Techniques for metadata processing

ABSTRACT

A method of and system for performing metadata tag compression in security policy enforcement system may comprise conveying a set of data elements, each with an associated metadata tag, from a first processor subsystem to a second processor subsystem. The first processor subsystem may be configured to process conventional tasks, the second processor configured to apply one or more policy decisions to the data element. The conveying may further comprise sending the set of data elements along with an index element that identifies one or more metadata tags, and sending one or more of the metadata tags identified by the index element.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Application Ser.No. 62/268,639 filed on Dec. 17, 2015, SOFTWARE DEFINED METADATAPROCESSING, DeHon, et al., and U.S. Provisional Application Ser. No.62/270,187 filed on Dec. 17, 2015, SOFTWARE DEFINED METADATA PROCESSING,DeHon, et al., both of which are entirely incorporated herein byreference.

BACKGROUND

This application relates generally to data processing and, moreparticularly, to programmable units for metadata processing.

Today's computer systems are notoriously hard to secure. Conventionalprocessor architectures, for example, allow various behaviors, such asbuffer overflows, pointer forging and the like, which violate higherlevel abstractions. Closing the gap between programming language andhardware may be left to software, where the cost of enforcing airtightabstractions is often deemed too high.

Some recent efforts have demonstrated the value of propagating metadataduring execution to enforce policies that catch safety violations andmalicious attacks as they occur. These policies can be enforced insoftware but typically incur high, undesirable overheads, such as inperformance and/or costs, that discourage their deployment or othermotivate coarse approximations providing less protection. Hardwaresupport for fixed policies may reduce overhead to acceptable levels andprevent a large portion of undesired code violations, such as may beperformed by malicious code or malware attacks. For example, Intelrecently announced hardware for bounds checking and isolation. Whilethese mitigate many of today's attacks, fully securing systems willrequire more than memory safety and isolation. Attacks rapidly evolve toexploit any remaining forms of vulnerability.

SUMMARY OF INVENTION

Thus, there is a need for a flexible security architecture that can bequickly adapted to this ever-changing landscape. It would be desirableto have such an architecture provide support for software-definedmetadata processing with minimal overhead. It is desirable that such anarchitecture is extensible to generally support and enforce any numberand type of policies without placing a visible, hard bound on the numberof bits allocated to metadata. Metadata may be propagated duringexecution to enforce policies and catch violation of such policies suchas, for example, by malicious code or malware attacks.

In accordance with one aspect of the techniques described herein is amethod of processing instructions comprising: receiving, for metadataprocessing, a current instruction with an associated metadata tag, saidmetadata processing being performed in a metadata processing domainisolated from a code execution domain including the current instruction;determining, in the metadata processing domain and in accordance withthe metadata tag and the current instruction, whether a rule exists in arule cache for the current instruction, said rule cache including ruleson metadata used by said metadata processing to define allowedoperations; and responsive to determining no rule exists in the rulecache for the current instruction, performing rule cache miss processingin the metadata processing domain comprising: determining whetherexecution of the current instruction is allowed; responsive todetermining the current instruction is allowed to be executed in thecode execution domain, generating a new rule for the currentinstruction; and responsive to writing to a register, inserting the newrule into the rule cache. First metadata used to select the rule for thecurrent instruction may be stored in a first portion of a plurality ofcontrol status registers used by the metadata processing. The firstportion of the plurality of control status registers may be used tocommunicate a plurality of metadata tags for the current instruction tothe metadata processing domain. The plurality of metadata tags may beused as data in the metadata processing domain. The register may be afirst control status register of a plurality of control status registersused by the metadata processing and a first portion of the plurality ofcontrol status registers may be used to communicate a plurality ofmetadata tags from the metadata processing domain to the rule cache. Theplurality of metadata tags may be for the current instruction. The newrule may be inserted into the rule cache responsive to writing a secondmetadata tag to the first control status register. The second metadatatag may be placed on a result of the current instruction and the resultmay be any of a destination register or a memory location. The pluralityof control status registers may include any one or more of: a bootstraptag control status register including an initial metadata tag from whichall other generated metadata tags are derived; a default tag controlstatus register specifying a default metadata tag; a public untrustedcontrol status register specifying a public untrusted metadata tag usedto tag instruction and data classified as public and untrusted; anopgroup value control status register including data written to a tableincluding information on opgroups and care information for differentopcodes; an opgroup address control status register specifying alocation in the table to which data of the opgroup value control statusregister is written; and a pumpflush control status register wherein awrite to the pumpflush control status register triggers flushing of therule cache. The plurality of control status registers may include a tagmode control status register denoting a current mode of metadataprocessing. The tag mode control status register may indicate whenmetadata processing is disengaged whereby rules of one or more definedpolicies are not enforced by metadata processing. The tag mode controlstatus register may be set to one of a defined set of allowed states todenote the current mode of metadata processing, the allowed statesincluding: an off state, a state whereby metadata processing writes adefault tag on all results, and a state indicating that metadataprocessing is engaged and operational when instructions are executed inthe code domain at one or more specified privilege levels. The rulecache miss processing may be performed in a first of the defined set ofallowed states where metadata processing is disengaged. The allowedstates may include a first state indicating that metadata processing isengaged only when instructions execute in the code domain at a userprivilege level; a second state indicating that metadata processing isengaged only when instructions execute in the code domain at a user orsupervisor privilege level; a third state indicating that metadataprocessing is engaged only when instructions execute in the code domainat a user, supervisor, or hypervisor privilege level; and a fourth stateindicating that metadata processing is engaged when instructions executein the code domain at a user, supervisor, hypervisor, or machineprivilege level. Whether the metadata processing is engaged ordisengaged may be determined in accordance with a current tag mode ofthe tag mode control status register in combination with a currentprivilege level of code executing in the code domain. Rules of one ormore defined policies may not be enforced when metadata processing isdisengaged and wherein the rules may be enforced when metadataprocessing is engaged. The plurality of control status registers mayinclude an extended opcode control status register including additionalopcode bits, if any, for the current instruction. The currentinstruction may be included in an instruction set having variable lengthopcodes. Each opcode of the instruction set may optionally include theadditional opcode bits. The extended opcode control status register mayinclude the additional opcode bits, if any, for the current instruction.A table may include information that maps an opcode of the instructionset to a corresponding opgroup and bit vector information. The opgroupmay denote a group of associated opcodes treated similarly by themetadata processing domain. The bit vector information may denotewhether particular inputs and outputs with respect to the metadataprocessing domain are used in connection with processing the opcode. Thetable may be indexed using a first portion of opcode bits less than amaximum number of allowable opcode bits. The maximum number may denotean upper bound on a number of bits of an opcode of the instruction set.For each opcode mapped using the table, there may be a result bit vectorcorresponding to the opcode. The result bit vector may denote whatportion, if any, of the additional opcode bits in the extended opcodecontrol status register are used with the opcode. The currentinstruction may be one of multiple instructions stored in a single wordof memory associated with a single metadata tag and the single metadatatag may be associated with the multiple instructions included in thesingle word. The plurality of control status registers may include asubinstruction control status register indicating which of the multipleinstructions stored in the single word is the current instruction. Thesingle metadata tag may be a first pointer to a first memory locationincluding a different metadata tag for each of the multiple instructionsin the single word. At least a first metadata tag stored in the firstmemory location for a first instruction of the multiple instructions mayinclude a second pointer to a second memory location including metadatatag information for the first instruction. The metadata tag informationfor the first instruction may include a complex structure where thecomplex structure may comprise at least one scalar data field and atleast one pointer field to a third memory location. A first memorylocation may include multiple bytes of data and a single metadata tagmay be associated with the first memory location. The first memorylocation may be a single word, and a plurality of control statusregisters used by the metadata processing may include a byte-enablecontrol status register indicating which of the multiple bytes stored inthe single word are any of read from, and written to, by the currentinstruction. The first memory location may be a single word, and aplurality of control status registers used by the metadata processingmay include a bit-enable control status register indicating which one ormore bits stored in the single word are any of read from, and writtento, by the current instruction. The single metadata tag may be a pointerto a second memory location including tags for the multiple bytes ofdata. The second memory location may include a variable length structurecomprising a first field denoting a number of subsequent tag fields ofthe variable length structure, and the subsequent tag fields may specifytags for the multiple bytes of data. The single metadata tag may be apointer to a hierarchical structure including tags for the multiplebytes of data.

In accordance with another aspect of techniques herein is anon-transitory computer readable medium comprising code thereon that,when executed, perform a method of processing instructions comprising:receiving, for metadata processing, a current instruction with anassociated metadata tag, said metadata processing being performed in ametadata processing domain isolated from a code execution domainincluding the current instruction; determining, in the metadataprocessing domain and in accordance with the metadata tag and thecurrent instruction, whether a rule exists in a rule cache for thecurrent instruction, said rule cache including rules on metadata used bysaid metadata processing to define allowed operations; and responsive todetermining no rule exists in the rule cache for the currentinstruction, performing rule cache miss processing in the metadataprocessing domain comprising: determining whether execution of thecurrent instruction is allowed; responsive to determining the currentinstruction is allowed to be executed in the code execution domain,generating a new rule for the current instruction; and responsive towriting to a register, inserting the new rule into the rule cache.

In accordance with another aspect of the techniques herein is a systemcomprising: a processor; and a memory comprising code stored thereonthat, when executed by the processor, performs a method of processinginstructions comprising: receiving, for metadata processing, a currentinstruction with an associated metadata tag, said metadata processingbeing performed in a metadata processing domain isolated from a codeexecution domain including the current instruction; determining, in themetadata processing domain and in accordance with the metadata tag andthe current instruction, whether a rule exists in a rule cache for thecurrent instruction, said rule cache including rules on metadata used bysaid metadata processing to define allowed operations; and responsive todetermining no rule exists in the rule cache for the currentinstruction, performing rule cache miss processing in the metadataprocessing domain comprising: determining whether execution of thecurrent instruction is allowed; responsive to determining the currentinstruction is allowed to be executed in the code execution domain,generating a new rule for the current instruction; and responsive towriting to a register, inserting the new rule into the rule cache. Theprocessor may be a pipeline processor in a reduced instruction setcomputing architecture.

BRIEF DESCRIPTION OF THE DRAWINGS

Features and advantages of the techniques herein will become moreapparent from the following detailed description of exemplaryembodiments thereof taken in conjunction with the accompanying drawingsin which:

FIG. 1 is a schematic drawing showing an example of a PUMP cacheintegrated as a pipeline stage in a processor pipeline;

FIG. 2 is a schematic drawing showing a PUMP Evaluation Framework;

FIG. 3A is a graph showing performance results for a single runtimepolicy with simple implementation using the evaluation frameworkdepicted in FIG. 2;

FIG. 3B is a graph showing performance results a single energy policywith simple implementation;

FIG. 4A is a series of bar graphs showing composite policy runtimeoverhead of simple implementation with 64b Tags, wherein the compositepolicy enforces simultaneously the following policies (i) spatial andtemporal memory safety, (ii) taint tracking, (iii) control-flowintegrity, and (iv) code and data separation;

FIG. 4B is a series of bar graphs showing composite policy energyoverhead of simple implementation with 64b Tags;

FIG. 4C is a series of bar graphs showing power ceilings with simpleimplementation compared to a baseline;

FIG. 5A is a comparative bar graph of the number of PUMP rules withoutopgroup optimization and with opgroup optimization;

FIG. 5B is a series of graphs showing the impact of miss rates ofdifferent opgroup optimizations based on PUMP capacity;

FIG. 6A is a graph of the distribution of unique tags for each DRAMtransfer for the gcc benchmark with the composite policy, showing thatmost words have the same tag;

FIG. 6B is a diagram showing the main memory tag compression;

FIG. 7A is a schematic drawings showing translation between 16b L2 tagsand 12b L1 tags;

FIG. 7B is a schematic drawings showing translation between 12b L1 tagsand 16b L2 tags;

FIG. 8A is a schematic graph showing the impact of L1 tag length on L1PUMP flushes (log 10);

FIG. 8B is a schematic graph showing the impact of L1 tag length on L1PUMP miss-rates;

FIG. 9A is a series of bar graphs showing miss rates for differentpolicies;

FIG. 9B is a line graph depicting a cache hit rate for four exemplarymicroarchitecture optimizations;

FIG. 9C is a line graph depicting miss service performance;

FIG. 9D is a line graph depicting miss handler hit rates based oncapacity;

FIG. 9E is a series of bar graphs depicting the impact of optimizationsfor the composite policy;

FIG. 10A is a series of graphs showing runtime overhead of optimizedimplementation;

FIG. 10B is a series of bar graphs showing energy overhead of optimizedimplementation;

FIG. 10C is a series of bar graphs showing absolute power of optimizedimplementation compared to a baseline;

FIG. 11A is a series of shaded graphs depicting runtime overhead impactof tag bit length and UCP-Cache ($) capacity for differentrepresentative benchmarks;

FIG. 11B is a series of shaded graphs depicting energy overhead impactof tag bit length and UCP-$ capacity for different representativebenchmarks;

FIG. 12A is a series of graphs showing runtime impact of optimizationson representative benchmarks wherein A: Simple; B: A+ Opgrouping; C:B+DRAM Compression; D: C+(10b L1, 14b, L2) short tags; E: D+(2048-UCP;512-CTAG));

FIG. 12B is a series of graphs showing energy impact of optimizations onrepresentative benchmarks wherein A: Simple; B: A+ Opgrouping; C: B+DRAMCompression; D: C+(10b L1, 14b, L2) short tags; E: D+(2048-UCP;512-CTAG));

FIG. 13A is a series of graphs showing runtime policy impact incomposition for a representative benchmark;

FIG. 13B is a series of graphs showing energy policy impact incomposition;

FIG. 14 is a first table labeled “TABLE 1” providing a summary ofinvestigated policies;

FIG. 15 is a second table labeled “TABLE 2” providing a summary oftaxonomy of tagging schemes;

FIG. 16 is a third table labeled “TABLE 3” providing a summary of memoryresource estimates for the baseline and the simple PUMP-extendedprocessor;

FIG. 17 is a fourth table labeled “TABLE 4” providing a summary of PUMPparameter ranges used in experiments;

FIG. 18 is a fifth table labeled “TABLE 5” providing a summary of memoryresource estimates for the PUMP-optimized processor;

FIG. 19 is a first algorithm labeled “Algorithm 1” providing a summaryof the taint tracking miss handler;

FIG. 20 is a second algorithm labeled “Algorithm 2” providing a summaryof the N-policy miss handler;

FIG. 21 a third algorithm labeled “Algorithm 3” providing a summary ofthe N-policy miss handler with HW support;

FIG. 22 is schematic view of the PUMP rule cache dataflow andmicroarchitecture;

FIG. 23 is a schematic view of the PUMP microarchitecture;

FIG. 24 is a schematic view, similar to FIG. 1, showing an exemplaryPUMP cache integrated as a pipeline stage in a processor pipeline andits opgroup translation, UCP and CTAG caches;

FIG. 25 is an example of control status registers (CSRs) in anembodiment in accordance with techniques herein;

FIG. 26 is an example of tagmodes in an embodiment in accordance withtechniques herein;

FIG. 27 is an example illustrating a separate metadata processingsubsystem/domain with a separate processor in an embodiment inaccordance with techniques herein;

FIG. 28 illustrates PUMP inputs and outputs in an embodiment inaccordance with techniques herein;

FIG. 29 illustrates inputs and outputs in connection with the opgrouptable that in an embodiment in accordance with techniques herein;

FIG. 30 illustrates processing performed by the PUMP in an embodiment inaccordance with techniques herein;

FIGS. 31 and 32 provide additional detail regarding control andselection of PUMP inputs and outputs in an embodiment in accordance withtechniques herein;

FIG. 33 is an example illustrating a 6 stage processing pipeline in anembodiment in accordance with techniques herein;

FIGS. 34-38 are examples illustrating subinstructions and associatedtechniques in an embodiment;

FIG. 36-38 are examples illustrating subinstructions and associatedtechniques in an embodiment;

FIGS. 39-42 are examples illustrating byte level tagging and associatedtechniques in an embodiment;

FIG. 43 is an example illustrating variable length opcodes in anembodiment in accordance with techniques herein;

FIG. 44 is an example illustrating an opcode mapping table in anembodiment in accordance with techniques herein;

FIG. 45 is an example illustrating shared pages in an embodiment inaccordance with techniques herein;

FIG. 46 is an example illustrating transfer of control points in anembodiment in accordance with techniques herein;

FIG. 47 is an example illustrating a call stack in an embodiment inaccordance with techniques herein;

FIGS. 48-49 are examples illustrating memory location tagging orcoloring in an embodiment in accordance with techniques herein;

FIG. 50 is an example illustrating setjmp and longjmp in an embodimentin accordance with techniques herein;

FIGS. 51, 52, and 53 are tables of different runtime behaviors andassociated preventive actions and mechanisms used to implement thepreventive actions in an embodiment in accordance with techniquesherein;

FIGS. 54, 55 and 56 are examples illustrating processing that may beperformed to learn or determine policy rules in an embodiment inaccordance with techniques herein;

FIGS. 57, 58, 59 and 60 are examples illustrating components in anembodiment in connection with converting between an external version andan internal tagged version of data;

FIGS. 61, 62 and 63 are examples illustrating aspects of performing tagprediction in an embodiment in accordance with techniques herein;

FIGS. 64-65 illustrate use of coloring memory location techniques hereinwith allocated memory in an embodiment;

FIG. 66-67 illustrate different components providing hardware rulesupport in an embodiment in accordance with techniques herein;

FIGS. 68-70 are examples illustrating use of techniques herein in anembodiment where the PUMP returns a value;

FIG. 71 is an example illustrating use of techniques herein in anembodiment with a sequence of instructions;

FIG. 72 is a flowchart of processing steps that may be performed inconnection with booting a system in an embodiment in accordance withtechniques herein;

FIG. 73 is an example of a tree tag hierarchy in connection with taggeneration in an embodiment in accordance with techniques herein;

FIGS. 74, 75, 76 and 77 are examples illustrating aspects and featuresin connection with an I/O PUMP in an embodiment in accordance withtechniques herein;

FIGS. 78, 79, 80, 81 and 82 are examples illustrating a hierarchy usedin connection with storing and determining tag values in an embodimentin accordance with techniques herein; and

FIGS. 83 and 84 are examples illustrating control flow integrity andassociated processing in an embodiment in accordance with techniquesherein.

DETAILED DESCRIPTION

Described in following paragraphs are various embodiments and aspects ofa Programmable Unit for Metadata Processing (PUMP) that indivisiblyassociates a metadata tag with every word in the system's main memory,caches, and registers. To support unbounded metadata, the tag is largeenough to indirect to a data structure in memory. On every instruction,the tags of the inputs are used to determine if the operation isallowed, and if so to calculate the tags for the results. In someembodiments, the tag checking and propagation rules are defined insoftware; however, to minimize performance impact, these rules arecached in a hardware structure, the PUMP rule cache, that operates inparallel with the arithmetic logic unit (ALU) portion of a processor. Insome embodiments, a miss handler, such as may be implemented usingsoftware and/or hardware, may be used to service cache misses based onthe policy currently in effect.

In at least one embodiment using a composition of four differentpolicies, the performance impact of the PUMP may be measured (See FIG.14) that stress the PUMP in different ways and illustrate a range ofsecurity properties, such as, for example, (1) a Non-Executable Data andNon-Writable Code (NXD+NWC) policy that uses tags to distinguish codefrom data in memory and provides protection against simple codeinjection attacks; (2) a Memory Safety policy that detects all spatialand temporal violations in heap-allocated memory, extending with aneffectively unlimited (260) number of colors (“taint marks”); (3) aControl-Flow Integrity (CFI) policy that restricts indirect controltransfers to only the allowed edges in a program's control flow graph,preventing return-oriented-programming-style attacks (fine grained CFIis enforced, not coarse-grained approximations that are potentiallyvulnerable to attack); and (4) a fine-grained Taint Tracking policy(generalizing) where each word can potentially be tainted by multiplesources (libraries and IO streams) simultaneously.

The foregoing are examples of well-known policies that may be used in anembodiment in accordance with techniques herein. For such well knownpolicies whose protection capabilities have been established in theliterature, techniques herein may be used to enforce such policies whilealso reducing the performance impact of enforcing them using the PUMP.Except for NXD+NWC, each of these policies needs to distinguish anessentially unlimited number of unique items; by contrast, solutionswith a limited number of metadata bits can, at best, support onlygrossly simplified approximations.

As illustrated and described elsewhere herein, one embodiment inaccordance with techniques herein may utilize a simple, directimplementation of the PUMP which uses pointer-sized (64b or byte) tagsto 64b words thereby at least doubling the size and energy usage of allthe memories in the system. Rule caches add area and energy on top ofthis. For this particular embodiment, an area overhead of 190% (See FIG.16) was measured and geomean energy overhead around 220%; moreover,runtime overhead may be over 300% on some applications. Such highoverheads may discourage adoption, if they were the best that could bedone.

However, as described in more detail below most policies exhibit spatialand temporal locality for both tags and the rules defined over them.Thus, an embodiment in accordance with techniques herein maysignificantly reduce the number of unique rules significantly bydefining them over a group of similar (or even identical) instructions,reducing compulsory misses and increasing the effective capacity of therule caches. Off-chip memory traffic can be reduced by exploitingspatial locality in tags. On-chip area and energy overhead can beminimized by using a small number of bits to represent the subset of thepointer-sized tags in use at a time. Runtime costs of composite policymiss handlers can be decreased by providing hardware support for cachingcomponent policies. Thus, an embodiment in accordance with techniquesherein may include such optimizations to thereby allow the PUMP toachieve lower overheads without compromising its rich policy model.

An embodiment in accordance with techniques herein may enhance memorywords and internal processor state with metadata that can be used toencode an arbitrary number of security policies that can be enforcedeither in isolation or simultaneously. An embodiment in accordance withtechniques herein may achieves the foregoing by adding, to a“conventional” processor (e.g. RISC-CPU, GPU, Vector processor, etc.), ametadata processing unit (the PUMP) that works in parallel with the dataflow to enforce an arbitrary set of policies; the present disclosuretechnique specifically makes the metadata unbounded and softwareprogrammable, such that the techniques herein may be adapted and appliedto a wide range of metadata processing policies. For example, the PUMPmay be integrated as a new/separate pipeline stage of a conventional(RISC) processor, or can be integrated as a stand-alone piece ofhardware working on parallel with the “host” processor. For the formercase, there may be an instruction level simulator, elaborated policies,implementation optimizations and resource estimates, and extensivesimulations to characterize the design.

Existing solutions trying to enforce policies at the fine (i.e.instruction) granularity level cannot enforce an arbitrary set ofpolicies. Commonly, only a small number of fixed policies can beenforced at the instruction level. Enforcing policies at a highergranularity level (i.e. thread) cannot prevent certain classes of ReturnOriented Programming attacks, thus rendering that type of enforcementlimited in its usefulness. In contrast, embodiments in accordance withtechniques herein allow the expression of an unlimited number ofpolicies (the only limit is the size address space, as the metadata isexpressed in terms of address pointers that can point to any arbitrarydata structures) that may be enforced at the instruction level singly orsimultaneously.

It should be noted that various figures described in followingparagraphs illustrate various examples, methods, and other exampleembodiments of various aspects of the techniques described herein. Itwill be appreciated that, in such figures, the illustrated elementboundaries (e.g., boxes, groups of boxes, or other shapes) generallyrepresent one example of the boundaries. One of ordinary skill in theart will appreciate that in some examples one element may be designed asmultiple elements or that multiple elements may be designed as oneelement. In some examples, an element shown as an internal component ofanother element may be implemented as an external component and viceversa. Furthermore, elements may not be drawn to scale.

Referring to FIG. 1, a Programmable Unit for Metadata Processing (PUMP)10 is integrated into a conventional Reduced Instruction Set Computingor Computer (RISC) processor 12 with an in-order implementation and a5-stage pipeline suitable for energy-conscious applications, whicheffectively transforms into a 6-stage pipeline with the addition of PUMP10. A first stage is a fetch stage 14, a second stage is a decode stage16, a third stage is an execute stage 18, a fourth stage is a memorystage 20, and a fifth stage is a writeback stage 22. Pump 10 isinterposed between the memory stage 20 and the writeback stage 22.

Various embodiments may implement the PUMP 10 using electronic logicthat is a mechanism providing policy enforcement and metadatapropagation. An embodiment of the PUMP 10 may be characterized by: (i)an empirical evaluation of the runtime, energy, power ceiling, and areaimpacts of a simple implementation of the PUMP 10 on a standard set ofbenchmarks under four diverse policies and their combination; (ii) a setof micro-architectural optimizations; and (iii) measurements of thegains from these optimizations, showing typical runtime overhead under10%, a power ceiling impact of 10%, and typically energy over-head under60% by using 110% additional area for on-chip memory structures.

In computing, benchmarking may be characterized as the act of running acomputer program, a set of programs, or other operations, in order toassess the relative performance of an object, normally by running anumber of standard tests and trials against it. The term ‘benchmark’used herein refers to benchmarking programs themselves. The types ofbenchmark programs used throughout this application and the Figures areGemsFDTD, astar, bwaves, bzip2, cactusADM, calculix, deall, gamess, gcc,gobmk, gromacs, h264ref, hmmer, Ibm, leslie3d, libquantum, mcf, mile,namd, omnetpp, perlbench, sjeng, specrand, sphinx3, wrf, zeusmp, andmean. See, for example, FIGS. 10A, 10B, and 10C.

“Logic”, as used herein, includes but is not limited to hardware,firmware, software and/or combinations of each to perform a function(s)or an action(s), and/or to cause a function or action from anotherlogic, method, and/or system. For example, based on a desiredapplication or needs, logic may include a software controlledmicroprocessor, discrete logic like a processor (e.g., microprocessor),an application specific integrated circuit (ASIC), a programmed logicdevice, a memory device containing instructions, an electric devicehaving a memory, or the like. Logic may include one or more gates,combinations of gates, or other circuit components. Logic may also befully embodied as software. Where multiple logics are described, it maybe possible to incorporate the multiple logics into one physical logic.Similarly, where a single logic is described, it may be possible todistribute that single logic between multiple physical logics.

In at least one embodiment in accordance with techniques herein, thePUMP 10 may be characterized as an extension to a conventional RISCprocessor 12. Following paragraphs provide further details of the ISA(instruction set architecture)-level extensions that constitute thePUMP's 10 hardware interface layer, the basic micro-architecturalchanges, and the accompanying low-level software that may be used in anembodiment in accordance with techniques herein.

In an embodiment in accordance with techniques herein, each word in aPUMP-enriched system may be associated with a pointer-sized tag. Thesetags are uninterpreted at the hardware level. At the software level, atag may represent metadata of unbounded size and complexity, as definedby the policy. Simpler policies that need only a few bits of metadatamay store the metadata directly in the tag; if more bits are required,then indirection is used to store the metadata as a data structure inmemory, with the address of this structure used as the tag. Notably,these pointer-sized tags are one exemplary aspect of the presentdisclosure and are not to be considered limiting. The basic addressablememory word is indivisibly extended with a tag, making all value slots,including memory, caches, and registers, suitably wider. A programcounter (PC) is also tagged. This notion of software-defined metadataand its representation as a pointer-sized tag extends previous taggingapproaches, where only a few bits are used for tags and/or they arehardwired to fixed interpretations. Some exemplary taxonomies of taggingschemes are presented in Table 2 which is reproduced in FIG. 15.

Metadata tags are not addressable by user programs. Rather, the metadatatags are addressed by policy handlers invoked on rule cache misses asdetailed below. All updates to tags are realized through PUMP 10 rules.

Besides unbounded metadata, another feature of an embodiment of the PUMP10 in accordance with techniques herein is hardware support forsingle-cycle common-case computation on metadata. These computations aredefined in terms of rules of the form opcode: (PC, CI, OP1, OP2,MR)⇒(PC_(new), R), which should be read: “If the current opcode isopcode, the current tag on the program counter is PC, the tag on thecurrent instruction is CI, the tags on its input operands (if any) areOP1 and OP2, and the tag on the memory location (in case of load/store)is MR, then the tag on the program counter in the next machine stateshould be PC_(new) and the tag on the instructions result (a destinationregister or a memory location, if any) should be R”. This rule format,allowing two output tags to be computed from up to five input tags, ismarkedly more flexible than those considered in prior work, whichtypically compute one output from up to two inputs (see Table 2 in FIG.15). Beyond previous solutions that only track data tags (OP1, OP2, MR,R), the present disclosure provides a current instruction tag (CI) thatcan be used to track and enforce provenance, integrity, and usage ofcode blocks; as well as a PC tag that can be used to record executionhistory, ambient authority, and “control state” including implicitinformation flows. The CFI policy exploits the PC tag for recording thesources of indirect jumps and the CI tag for identifying jump targets,NXD+NWC leverages the CI to enforce that data is not executable, andTaint Tracking uses the CI to taint data based on the code that producedit.

To resolve the rules in a single cycle in the common case, an embodimentin accordance with techniques herein may use a hardware cache of themost recently used rules. Depending on the instruction and policy, oneor more of the input slots in a given rule may be unused. To avoidpolluting the cache with rules for all possible values of the unusedslots, the rule-cache lookup logic refers to a bit vector containing a“don't-care” (See FIG. 1) bit for each input slot-opcode pair, whichdetermines whether the corresponding tag is actually used in the rulecache lookup. To handle these “don't care” inputs efficiently, they aremasked out before presenting the inputs to the PUMP 10. The don't-carebit vectors are set by a privileged instruction as part of the misshandler installation.

FIG. 1 generally illustrates one embodiment in accordance withtechniques herein with a revised 5-stage processor 12 pipeline thatincorporates the PUMP 10 hardware. The rule cache lookup is added as anadditional stage and bypass tag and data independently so that the PUMP10 stage does not create additional stalls in the processor pipeline.

Placing the PUMP 10 as a separate stage (between memory stage 20 andwriteback stage 22) is motivated by the need to provide the tag on theword read from memory (load), or to be overwritten in memory (store), asan input to the PUMP 10. Since rules are allowed that depend on theexisting tag of the memory location that is being written, writeoperations become read-modify-write operations. The existing tag is readduring the Memory stage 20 like a read rule, the read rule is checked inthe PUMP 10 stage, and the write is performed during the Commit stagewhich may also be referred to as writeback stage 22. As with any cachingscheme, multiple levels of caches may be used for the PUMP 10. Asdescribed in more detail below, an embodiment in accordance withtechniques herein may utilize two levels of caches. The extension tomultiple levels of caches is readily apparent to one having ordinaryskill in the art.

In one non-limiting example, when a last-level miss occurs in the rulecache in the writeback stage 22, it is handled as follows: (i) thecurrent opcode and tags are saved in a (new) set of processor registersused only for this purpose, and (ii) control is transferred to thepolicy miss handler (described in more detail below), which (iii)decides if the operation is allowed and if so generates an appropriaterule. When the miss handler returns, the hardware (iv) installs thisrule into the PUMP 10 rule caches, and (v) re-issues the faultinginstruction. To provide isolation between the privileged miss handlerand the rest of the system software and user code, a miss-handleroperational mode is added to the processor, controlled by a bit in theprocessor state that is set on a rule cache miss and reset when the misshandler returns. To avoid the need to save and restore registers onevery rule cache miss, the integer register file may be expanded with 16additional registers that are available only to the miss handler.Additionally, the rule inputs and outputs appear as registers while inmiss handler mode (cf. register windows), allowing the miss handler (butnothing else) to manipulate the tags as ordinary values. Again, theseare all non-limiting examples of the writeback stage 22.

A new miss-handler-return instruction is added to finish installing therule into the PUMP 10 rule caches and returns to user code. In thisparticular non-limiting example, this instruction can only be issuedwhen in miss-handler mode. While in miss-handler mode, the rule cache isignored and the PUMP 10 instead applies a single, hardwired rule: allinstructions and data touched by the miss handler must be tagged with apredefined MISSHANDLER tag, and all instruction results are given thesame tag. In this way, the PUMP 10 architecture prevents user code fromundermining the protection provided by the policy. Alternatively, thePUMP may be used to enforce flexible rules on miss-handler access. Tagsare not divisible, addressable, or replaceable by user code; metadatadata structures and miss handler code cannot be touched by user code;and user code cannot directly insert rules into the rule cache.

With reference to FIG. 19, Algorithm 1 illustrates the operation of themiss handler for a taint-tracking policy. To minimize the number ofdistinct tags (and hence rules), the miss handler uses a single tag forlogically equivalent metadata by “canonicalizing” any new datastructures that it builds.

Rather than forcing users to choose a single policy, multiple policiesare enforced simultaneously and new ones are added later. An exemplaryadvantage to these “unbounded” tags is that they can enforce any numberof policies at the same time. This can be achieved by letting tags bepointers to tuples of tags from several component policies. For example,to combine the NXD+NWC policy with the taint-tracking policy, each tagcan be a pointer to a tuple (s, t), where s is a NXD+NWC tag (eitherDATA or CODE) and t is a taint tag (a pointer to a set of taints). Therule cache lookup is similar, however when a miss occurs, both componentpolicies are evaluated separately: the operation is allowed only if bothpolicies allow it, and the resulting tags are pairs of results from thetwo component policies. However, in other embodiments, it might bepossible to express how the policies are to be combined (not simply asAND between all the constituent components).

With reference to FIG. 20, Algorithm 2 illustrates the general behaviorof the composite miss handler for any N policies. Depending on howcorrelated the tags in the tuple are, this could result in a largeincrease in the number of tags and hence rules. In order to demonstratethe ability to support multiple policies simultaneously and measure itseffect on working set sizes, a composite policy (“Composite”) wasimplemented through experimentation and where the composite policycomprises all four policies described above. The Composite policyrepresents the kind of policy workloads that are supported which aredescribed in further detail below. As seen in FIG. 4A and FIG. 20, thecomposite policy enforces simultaneously the following policies (i)spatial and temporal memory safety, (ii) taint tracking, (iii)control-flow integrity, and (iv) code and data separation

Most policies will dispatch on the opcode to select the appropriatelogic. Some policies, like NXD+NWC, will just check whether theoperation is allowed. Others may consult a data structure (e.g., the CFIpolicy consults the graph of allowed indirect call and return ids).Memory safety checks equality between address color (i.e pointer color)and memory region colors. Taint tracking computes fresh result tags bycombining the input tags (Alg. 1). Policies that must access large datastructures (CFI) or canonicalize across large aggregates (TaintTracking, Composite) may make many memory accesses that will miss in theon-chip caches and go to DRAM. On average across all of the benchmarks,servicing misses for NXD+NWC required 30 cycles, Memory Safety required60 cycles, CFI required 85 cycles, Taint Tracking required 500 cycles,and Composite required 800 cycles.

If the policy miss handler determines that the operation is not allowed,it invokes a suitable security fault handler. What this fault handlerdoes is up to the runtime system and the policy; typically, it wouldshut down the offending process, but in some cases it might return asuitable “safe value” instead. For incremental deployment withUNIX-style operating systems, assumed policies are applied per process,allowing each process to get a different set of policies. The recitationof being applied per process is non-limiting but rather exemplary andone having skill in the art recognizes this. It also allows us to placethe tags, rules, and miss handling support into the address space of theprocess, avoiding the need for an OS-level context switch. Longer term,perhaps PUMP policies can be used to protect the OS as well.

The following details evaluation methodology for measuring runtime,energy, area, and power overheads and applies it on a simpleimplementation of the PUMP hardware and software, using 128b words (64bpayload and 64b tag) and the modified pipeline processor 12 depicted inFIG. 1. It is useful to describe and measure the simple PUMPimplementation first, even though the optimized implementation is theversion to which the overheads (relative to the baseline processor) isultimately desired. Both are described because it details basic versionsof the key mechanisms before getting to more sophisticated versions.

To estimate the physical resource impact of the PUMP, memory costs wereprimarily focused on, since the memories are the dominant area andenergy consumers in a simple RISC processor and in the PUMP hardwareextensions. A 32 nm Low Operating Power (LOP) process is considered forthe L1 memories (See FIG. 1) and Low Standby Power (LSTP) for the L2memories and use CACTI 6.5 for modeling the area, access time, energyper access, and static (leakage) power of the main memory and theprocessor on-chip memories.

A baseline processor (no-PUMP) has separate 64 KB L1 caches for data andinstructions and a unified 512 KB L2 cache. Delay-optimized L1 cachesand an energy-optimized L2 cache were used. All caches use a writebackdiscipline. The baseline L1 cache has a latency around 880 ps; it isassumed that it can return a result in one cycle and set its clock to 1ns, giving a 1 GHz-cycle target-comparable to modern embedded and cellphone processors. The parameters for this processor are presented inTable 3 in FIG. 16.

One embodiment of the PUMP rule cache 10 hardware implementation mayinclude two parts: extending all architectural states in stages 14, 16,20 with tags, and adding PUMP rule caches to the processor 12. Extendingeach 64b word in the on-chip memories with a 64b tag increases theirarea and energy per access and worsens their access latency. This ispotentially tolerable for the L2 cache, which already has a multi-cycleaccess latency and is not used every cycle. But adding an extra cycle oflatency to access the L1 caches (See FIG. 1) can lead to stalls in thepipeline. To avoid this, in this simple implementation the effectivecapacity of the L1 caches is reduced to half of those in the baselinedesign and then add tags; this gives the same single-cycle access to theL1 caches, but can degrade performance due to increased misses.

In an embodiment in accordance with techniques herein, the PUMP rulecache 10 utilizes a long match key (5 pointer-sized tags plus aninstruction opcode, or 328 b) compared to a traditional cache addresskey (less than the address width), and returns a 128b result. In oneembodiment, a fully associative L1 rule cache may be used but would leadto high energy and delay (See Table 3 in FIG. 16). As an alternative, anembodiment in accordance with techniques herein may utilize a multi-hashcache scheme inspired with four hash functions, as depicted in FIG. 22.The L1 rule cache is designed to produce a result in a single cycle,checking for a false hit in the second cycle, while the L2 rule cache isdesigned for low energy, giving a multi-cycle access latency. Again,Table 3 in FIG. 16 shows the parameters for 1024-entry L1 and 4096-entryL2 rule caches used in the simple implementation. When these cachesreach capacity, a simple first-in-first out (FIFO) replacement policy isused, which appears to work well in practice for the current workloads(FIFO is within 6% of LRU here).

With reference to FIG. 2, the estimation of the performance impact ofthe PUMP identifies a combination of ISA, PUMP, and address-tracesimulators. A gem5 simulator 24 generates instruction traces for theSPEC CPU2006 programs (omitting xalancbmk and tonto, on which gem5fails) on a 64-bit Alpha baseline ISA. Each program simulates for eachof the four policies listed above and the composite policy for a warm-upperiod of 1B instructions and then evaluates the next 500M instructions.In gem5 simulator 24, each benchmark is run on the baseline processorwith no tags or policies. The resulting instruction trace 26 is then runthrough a PUMP simulator 28 that performs metadata computation for eachinstruction. This “phased” simulation strategy is accurate for fail-stoppolicies, where the PUMP's results cannot cause a program's control flowto diverge from its baseline execution. While address-trace simulationscan be inaccurate for highly pipelined and out-of-order processors, theyare quite accurate for the simple, in-order, 5- and 6-stage pipeline. Onthe baseline configuration, the gem5 instruction simulation and addresstrace generation 30 followed by custom address-trace simulations inaddress simulator 32 and accounting were within 1.2% of gem5'scycle-accurate simulations.

The PUMP simulator 28 includes miss-handler code (written in C) toimplement each policy, and metadata tags are assigned on the initialmemory depending on the policy. The PUMP simulator 28 allows capturesthe access patterns in the PUMP 10 rule caches and estimates theassociated runtime and energy costs, accounting for the longer waitcycles required to access the L2 rule cache. Since the PUMP simulator 28having miss handler code also runs on the processor, separatesimulations for the miss handler on gem5 to capture its dynamicbehavior. Since the miss-handler code potentially impacts the data andinstruction caches, a merged address trace is created that includesproperly interleaved memory accesses from both user and miss-handlercode, which is used for the final address-trace simulation to estimatethe performance impact of the memory system.

In following paragraphs, the evaluation of the simple PUMPimplementation is provided in comparison to the no-PUMP baseline.

As one point of evaluation, it should be noted that the overall areaoverhead of the PUMP 10 on top of the baseline processor is 190% (SeeTable 3 in FIG. 16). The dominant portion of this area overhead (110%)comes from the PUMP 10 rule caches. The unified L2 cache contributesmost of the remaining area overhead. The L1 D/I caches stay roughly thesame, since their effective capacity is halved. This high memory areaoverhead roughly triples the static power, contributing to 24% of theenergy overhead.

Another points of evaluation relates to runtime overhead. For all singlepolicies on most benchmarks, the average runtime overhead of even thissimple implementation is only 10% (see FIG. 3A and FIG. 3B; to readboxplots: bar is the median, box covers one quartile above and below(middle 50% of cases), dots represent each individual data point,whiskers denote full range except for outliers (more than 1.5×respective quartile)), with the dominant overhead coming from theadditional DRAM traffic required to transfer tag bits to and from theprocessor. For the Memory Safety policy (FIG. 3A and FIG. 3B), there area few benchmarks that exhibit high miss handler overhead, pushing theirtotal overhead up to 40-50% due to compulsory misses on newly allocatedmemory blocks. For the Composite policy Runtime (labeled as “CPI” or“CPI Overhead” in the Figures), five of the benchmarks suffer from veryhigh overheads in the miss handler (See FIG. 4A), with the worst caseclose to 780% in the GemsFTDT and the geomean reaching 50%. For theComposite policy Energy (labeled as “EPI” or “EPI Overhead” in theFigures) depicted in FIG. 4B, three of the benchmarks (i.e. GemsFTDT,astar, omnetpp) suffer from very high overheads in the miss handler,with the worst case close to 1600% in the GemsFTDT, 600% in the astar,and 520% in the omnetpp.

Two factors contribute to this overhead: (1) the large number of cyclesrequired to resolve a last-level rule cache miss (since every componentmiss handler must be consulted), and (2) an explosion in the number ofrules, which expands the working set size and increases the rule cachemiss rate. In the worst case, the number of unique composite tags couldbe the product of the unique tags in each component policy. However, thetotal rules increase by a factor of 3×-5× over the largest singlepolicy, Memory Safety.

Another point of evaluation is energy overhead. Moving more bits, due towider words, and executing more instructions, due to miss handler code,both contribute to energy overheads, impacting both the single andcomposite policies (FIG. 3B and FIG. 4B). The CFI and Memory Safetypolicies—and hence also the Composite policy—access large datastructures that often require energy-expensive DRAM accesses. Theworst-case energy overhead is close to 400% for single policies, andabout 1600% for the Composite policy, with geomean overhead around 220%.

For many platform designs the worst-case power, or equivalently, energyper cycle, is the limiter. This power ceiling may be driven by themaximum current the platform can draw from a battery or the maximumsustained operating temperature either in a mobile or in a wired devicewith ambient cooling. FIG. 4C shows that the simple implementationraises the maximum power ceiling by 76% with lbm driving the maximumpower in both the baseline and simple PUMP implementations. Note thatthis power ceiling increase is lower than the worst-case energy overheadin part because some benchmarks slow down more than the extra energythey consume and in part because the benchmarks with high energyoverhead are the ones consuming the least absolute energy per cycle inthe baseline design. Typically the data working set of theseenergy-efficient programs fits into the on-chip caches, so they seldompay the higher cost of DRAM accesses.

An embodiment incorporating the foregoing implementation described aboveachieves reasonable performance on most benchmarks, the runtime overheadfor the Composite policy on some of them and the energy and poweroverheads on all policies and benchmarks seem unacceptably high. Toaddress these overheads, a series of targeted microarchitectureoptimizations may be introduced and also incorporated into an embodimentin accordance with techniques herein. In Table 4 at FIG. 17, theseoptimizations are examined for the impact of the architecturalparameters associated with the PUMP components on the overall costs.Groupings of opcodes with identical rules are used to increase theeffective capacity of the PUMP rule caches, tag compression to reducethe delay and energy of DRAM transfers, short tags to reduce the areaand energy in on-chip memories, and Unified Component Policy (UCP) andComposition Tag (CTAG) caches to decrease the overheads in the misshandlers.

What will now be described are “opgroups” as may be used in anembodiment in accordance with techniques herein. In practical policies,it is common to define similar rules for several opcodes. For example,in the Taint Tracking policy, the rules for the Add and Sub instructionsare identical (See Algorithm 1 in FIG. 19). However, in the simpleimplementation, these rules occupy separate entries in the rule caches.Based on this observation, instruction operation codes (“opcodes”) aregrouped with the same rules into “opgroups”, reducing the number ofrules needed. Which opcodes can be grouped together depends on thepolicy; therefore the “don't-care” SRAM is expanded in the Execute stage18 (FIG. 1) to also translate opcodes to opgroups before the rule cachelookup. For the Composite policy, over 300 Alpha opcodes are reduced to14 opgroups and the total number of rules by a factor of 1.1×-6×, withan average of 1.5× (FIG. 5A measures this effect across all the SPECbenchmarks). This effectively increases the rule cache capacity for agiven investment in silicon area. Opgroups also reduce the number ofcompulsory misses, since a miss on a single instruction in the groupinstalls the rule that applies to every instruction opcode in the group.FIG. 5B summarizes the miss-rate across all the SPEC benchmarks fordifferent L1 rule cache sizes for the Composite policy with and withoutopgrouping. FIG. 5B shows that both the range and the mean of themiss-rates are reduced by opgrouping. Particularly, a 1024-entry rulecache after opgroup optimization has a lower miss rate than a 4096-entryrule cache without it. A lower miss-rate naturally reduces the time andenergy spent in miss handlers (See FIG. 12A and FIG. 12B) and smallerrule caches directly reduce area and energy.

An embodiment in accordance with techniques herein may utilize mainmemory tag compression that will now be described. Using 64b tags on 64bwords doubles the off-chip memory traffic and therefore approximatelydoubles the associated energy. Typically, though, tags exhibit spatiallocality—many adjacent words have the same tag. For example, FIG. 6Aplots the distribution of unique tags for each DRAM transfer for the gccbenchmark with the Composite policy, showing that most words have thesame tag: on average there are only about 1.14 unique tags per DRAMtransfer of an 8-word cache line. This spatial tag locality is exploitedto compress the tag bits that must be transferred to and from theoff-chip memory. Since data is transferred in cache lines, the cachelines are used as the basis for this compression. 128B per cache lineare allocated in the main memory, to keep addressing simple.

However, as depicted in FIG. 6B rather than storing 128b tagged wordsdirectly, eight 64b words (payloads) are stored, followed by eight 4bindexes, and then up to eight 60b tag. The index identifies which of the60b tags goes with the associated word. The tag is trimmed to 60b toaccommodate the indexes, but this does not compromise the use of tags aspointers: assuming byte addressing and 16B (two 64b words) alignedmetadata structures, the low 4b of the 64b pointer can be filled in aszeros. As a result, after transferring the 4B of indexes, all thatremains is the need to transfer the unique 7.5B tags in the cache line.For instance, if the same tag is used by all the words in the cache linethen there is a transfer of 64B+4B=68B in a first read, then 8B in asecond read for a total of 76B instead of 128B. The 4b index can beeither a direct index or a special value. A special index value isdefined to represent a default tag, so that there is no need to transferany tag in this case. By compressing tags in this manner, the averageenergy overhead per DRAM transfer is reduced from 110% to 15%.

The compression scheme presented above may be utilized in embodiment inaccordance with techniques herein, for example, due to its combinationof simplicity and effectiveness at reducing off-chip memory energy. Onehaving skill in the art clearly recognizes that additional alternativeclever schemes for fine-grained memory tagging exist—includingmulti-level tag page tables, variable-grained TLB-like structures, andrange caches—and these may also be used to reduce the DRAM footprint inan embodiment in accordance with techniques herein.

What will now be described is how tag translation may be performed in anembodiment in accordance with techniques herein. With reference again toFIG. 1, the simple PUMP rule caches are large (adding 110% area) sinceeach cached rule is 456b wide. Supporting the PUMP 10 also requiredextending the baseline on-chip memories (RFs and L1/L2 caches) with 64btags. Using a full 64b (or 60b) tag for each 64b word here incurs heavyarea and energy overheads. However, a 64 KB L1-D$ holds only 8192 wordsand hence at most 8192 unique tags. Along with a 64 KB L1-I$, there maybe at most 16384 unique tags in the L1 memory subsystem; these can berepresented with just 14b tags, reducing the delay, area, energy, andpower in the system. Caches (L1, L2) exist to exploit temporal locality,and this observation suggests that locality can be leveraged to reducearea and energy. If the tag bits are reduced to 14b, the PUMP rule cachematch key is reduced from 328 b to 78 b.

To obtain the foregoing saving advantage without losing the flexibilityof full, pointer-sized tags, different-width tags may be used fordifferent on-chip memory subsystems and translate between these asneeded. For example, one might use 12b tags in the L1 memories and 16btags in the L2 memories. FIG. 7A details tag translation as may beperformed between L1 and L2 memory subsystems. Moving a word from L2cache 34 to L1 cache 36 requires translating its 16 b tag to thecorresponding 12b tag, creating a new association if needed. A simpleSRAM 38 for the L2-tag-to-L1-tag translation, with an extra bitindicating whether or not there is an L1 mapping for the L2 tag. FIG. 7Bdetails the translation an L1 tag 40 to L2 tag 42 (on a writeback or anL2 lookup) performed with a SRAM 39 lookup using the L1 tag as theaddress. A similar translation occurs between the 60b main memory tagsand 16b L2 tags.

When a long tag is not in the long-to-short translation table, a newshort tag is allocated, potentially reclaiming a previously allocatedshort tag that is no longer in use. There is a rich design space toexplore for determining when a short tag can be reclaimed, includinggarbage collection and tag-usage counting. For simplicity, short tagsare allocated sequentially and flush all caches above a given level(instruction, data, and PUMP) when the short tag space is exhausted,avoiding the need to track when a specific short tag is available forreclamation. Caches may be designed with suitable techniques that makingcache flushes inexpensive. For example, in an embodiment in accordancewith techniques herein, all caches may be designed with a lightweightgang clear, such as known in the art and described, for example, in K.Mai, R. Ho, E. Alon, D. Liu, Y. Kim, D. Patil, and M. Horowitz.Architecture and Circuit Techniques for a 1.1 GHz 16-kb ReconfigurableMemory in 0.18 um-CMOS. IEEE J. Solid-State Circuits, 40(1):261-275,January 2005, which is incorporated by reference herein.

In comparison to Table 3 (reproduced in FIG. 16), where each L1 rulecache access costs 51 pJ, techniques herein provide for a reduction downto 10 pJ with 8b L1 tags or 18 pJ with 16b L1 tags, with the energyscaling linearly with tag length between these points. The energy impacton the L1 instruction and data caches is small. Similarly, with 16b L2tags, L2 PUMP access costs 120 pJ, down from 173 pJ with 64b tags.Slimming L1 tags also allows us to restore the capacity of the L1caches. With 12b tags, the full-capacity (76 KB, effective 64 KB) cachewill meet single-cycle timing requirements, reducing the performancepenalty the simple implementation incurred from the reduced L1 cachecapacity. As a result, L1 tag length exploration is limited to 12 bitsor less. While even shorter tags reduce energy, they also increase thefrequency of flushes.

FIG. 8A and FIG. 8B depict how flushes decrease with increasing L1 taglength, as well as the impact on the L1 rule cache miss-rate.

What will now be described are various techniques that may be used inconnection with miss handler acceleration. An embodiment in accordancewith techniques herein may combine four policies into a single Compositepolicy. With reference to FIG. 20, in Algorithm 2, each invocation of aN-policy miss handler has to take apart a tuple of tags and rules neededfor the Composite policy increases the rule cache miss rates, which areidentified in FIG. 9A. Even though the Taint Tracking and CFI policiesindividually have a low miss-rate, a higher miss-rate from the MemorySafety policy drives the miss-rate for the Composite policy high aswell. The lower miss rates of the individual policies suggest that theirresults may be cacheable even when the composite rules are not.

In connection with various aspects of the PUMP microarchitecture such asillustrated in FIG. 23, hardware structures may be utilized to optimizecomposite policy miss handling. An embodiment in accordance withtechniques herein may utilize a Unified Component Policy (UCP; seeAlgorithm 3 in FIG. 21) cache (UCP $) where the most recent componentpolicy results are cached. In such an embodiment, the generalmiss-handler for composite policies is modified to perform lookups inthis cache while resolving component policies (e.g., see Algorithm 3 ofFIG. 21, such at line 3). When this cache misses for a component policyits policy computation is performed in software (and insert the resultin this cache).

As also illustrated in FIG. 24, the UCP cache may be implemented withthe same hardware organization as the regular PUMP rule cache, with anadditional policy identifier field. A FIFO replacement policy may beused for this cache, but it may be possible to achieve better results byprioritizing space using a metric such as the re-computation cost forthe component policies. With modest capacity, this cache filters outmost policy re-computations (FIG. 9B; the low hit rate for memory safetyis driven by compulsory misses associated with new memory allocations).As a result, the average number of miss handler cycles are reduced by afactor of 5 for the most challenging benchmarks (FIG. 9E). It ispossible for every policy to hit in the UCP cache when there is a missin the L2 PUMP since the composite rules needed could be a product of asmall number of component policy rules. For GemsFDTD, three or morecomponent policies was hit about 96% of the time.

As also included in FIG. 23 and FIG. 24, a cache may be added totranslate a tuple of result tags into its canonical composite resulttag. The foregoing cache may be referred to as the Composition Tag(CTAG) cache (CTAG $) which is effective (FIG. 9D) because it is commonfor several component policy rules to return the same tuple of resulttags. For example, in many cases the PC_(tag) will be the same, eventhough the result tag is different. Furthermore, many different ruleinputs can lead to the same output. For example, in Taint Tracking setunions are performed, and many different unions will have the sameresult; e.g., (Blue, {A, B, C}) is the composite answer for writing theresult of both {A} U {B,C} and {A, B} U {B,C} (Taint Tracking) into aBlue slot (Memory Safety). A FIFO replacement policy is used for thiscache. The CTAG cache reduces the average miss handler cycles by anotherfactor of 2 (See FIG. 9E).

Taken together, a 2048-entry UCP cache and a 512-entry CTAG cache reducethe average time spent on each L2 rule cache miss from 800 cycles to 80cycles.

An embodiment in accordance with techniques herein may also improveperformance by prefetching one or more rules which are stored in one ormore of the caches including rules. Thus, it is additionally possible toreduce the compulsory miss rate with precompute rules that might beneeded in the near future. An exemplary instance has high value for theMemory Safety rules. For example when a new memory tag is allocated, newrules will be needed (initialize (1), add offset to pointer and move(3), scalar load (1), scalar store (2)) for that tag. Consequently, allof these rules may be added to the UCP cache at once. For thesingle-policy Memory Safety case, the rules may be added directly intothe rule caches. This reduces the number of Memory Safety miss-handlerinvocations by 2×.

In connection with an overall evaluation and with reference to FIG. 11A,the architecture parameters monotonically impact a particular cost,providing tradeoffs among energy, delay, and area, but not defining aminimum within a single cost criteria. There is the threshold effectthat, once the tag bits are small enough, the L1 D/I caches can berestored to the capacity of the baseline, so that baseline is adopted asthe upper bound to explore for L1 tag length, but beyond that point,decreasing tag length reduces energy with small impact on performance.

FIG. 11B depicts that reducing tag length is the dominant energy effectfor most benchmark programs (e.g. leslie3d, mcf), with a few programsshowing equal or larger benefits from increasing UCP cache capacity(e.g., GemsFDTD, gcc). Ignoring other cost concerns, to reduce energy,large miss handler caches and few tag bits are selected. Runtimeoverhead (see FIG. 11A) is also minimized with larger miss handlercaches, but benefits from more rather than fewer tag bits (e.g.,GemsFDTD, gcc). The magnitude of the benefits vary across benchmarks andpolicies. Across all benchmarks, the benefit beyond 10b L1 tags is smallfor the SPEC CPU2006 benchmarks, so 10b are used as the compromisebetween energy and delay and use a 2048-entry UCP cache and a 512-entryCTAG cache to reduce area overhead while coming close to the minimumenergy level within the space of the architecture parameters explored.

FIG. 12A and FIG. 12B depict the overall impact on runtime and energyoverheads of applying the optimizations. Every optimization is dominantfor some benchmark (e.g., opgroups for astar, DRAM tag compression forlbm, short tags for h264ref, miss handler acceleration for GemsFDTD),and some benchmarks see benefits from all optimizations (e.g. gcc), witheach optimization successively removing one bottleneck and exposing thenext. The different behavior from the benchmarks follows their baselinecharacteristics as detailed below.

Applications with low locality have baseline energy and performancedriven by DRAM due to high main memory traffic. The overhead in suchbenchmarks (e.g., lbm) trends to the DRAM overhead, so reductions inDRAM overhead directly impact runtime and energy overhead. Applicationswith more locality are faster in the baseline configuration, consumeless energy, and suffer less from DRAM overheads; as a result, thesebenchmarks are more heavily impacted by the reduced L1 capacity and thetag energy in the L1 D/I and rule caches. DRAM optimization has lesseffect on these applications, but using short tags has a large effect onenergy and removes the L1 D/I cache capacity penalty (e.g. h264ref).

The benchmarks with heavy dynamic memory allocation have higher L2 rulecache miss rates due to compulsory misses as newly created tags must beinstalled in the cache. This drove the high overheads for severalbenchmarks (GemsFDTD, omnetpp) in the simple implementation. The misshandler optimizations as described herein reduce the common case cost ofsuch misses, and the opgroup optimization reduces the capacity missrate. For the simple implementation, GemsFDTD took an L2 rule cache missevery 200 instructions and took 800 cycles to service each miss drivinga large part of its 780% runtime overhead (See FIG. 4A). With theoptimizations, the GemsFDTD benchmark services an L2 rule cache missevery 400 instructions and takes only 140 cycles on average per miss,reducing its runtime overhead to about 85% (See FIG. 10A).

Overall, these optimizations bring runtime overhead below 10% for allbenchmarks except GemsFDTD and omnetpp (See FIG. 10A), which are high onmemory allocation. The mean energy overhead is close to 60%, with only 4benchmarks exceeding 80% (See FIG. 10B).

To illustrate, the performance impact of the PUMP may be measured usinga composition of four different policies (See Table 1 at FIG. 14) thatstress the PUMP in different ways and illustrate a range of securityproperties: (1) a Non-Executable Data and Non-Writable Code (NXD+NWC)policy that uses tags to distinguish code from data in memory andprovides protection against simple code injection attacks; (2) a MemorySafety policy that detects all spatial and temporal violations inheap-allocated memory, extending with an effectively unlimited (260)number of colors (“taint marks”); (3) a Control-Flow Integrity (CFI)policy that restricts indirect control transfers to only the allowededges in a program's control flow graph, preventingreturn-oriented-programming-style attacks (enforce fine-grained CFI, notcoarse-grained approximations that are potentially vulnerable toattack); and (4) a fine-grained Taint Tracking policy (generalizing)where each word can potentially be tainted by multiple sources(libraries and IO streams) simultaneously. As noted elsewhere herein,these are well-known policies whose protection capabilities have beenestablished in the literature and description herein may focus onmeasuring and reducing the performance impact of enforcing them usingthe PUMP. Except for NXD+NWC, each of these policies distinguishes anessentially unlimited number of unique items; by contrast, solutionswith a limited number of metadata bits can, at best, support onlygrossly simplified approximations. As also noted above, a simple, directimplementation of the PUMP may be expensive. For example, addingpointer-sized (64b) tags to 64b words at least doubles the size andenergy usage of all the memories in the system; rule caches add area andenergy on top of this. For this simple implementation, the measured areaoverhead is about 190% and geomean energy overhead is around 220%;moreover, runtime overhead is disappointing (over 300%) on someapplications. Such high overheads would discourage adoption, if theywere the best that could be done.

The micro-architecture optimizations such as described herein may beincluded in an embodiment in accordance with techniques herein to reducethe impact on power ceiling to 10% (See FIG. 10C), suggesting theoptimized PUMP will have little impact on the operating envelope of theplatform. DRAM compression reduces the energy overhead for lbm to 20%;since it also slows down by 9%, its power requirement only increases by10%.

The area overhead of the optimized design is around 110% (e.g., seeTable 5 of FIG. 18) in comparison to 190% of the simple design (e.g.,see Table 3 of FIG. 16). Short tags significantly reduce the area of theL1 and L2 caches (now adding only 5% over the baseline) and of the rulecaches (adding only 26%). Contrarily, the optimized design spends somearea to reduce runtime and energy overhead. The UCP and CTAG caches add33% area overhead, while the translation memories for short tags (bothL1 and L2) add another 46%. While these additional hardware structuresadd area, they provide a net reduction in energy, since they areaccessed infrequently and the UCP and CTAG caches also substantiallyreduce the miss-handler cycles.

One goal of the model and optimizations as described herein is to makeit relatively simple for an embodiment to add additional policies thatare simultaneously enforced. The Composite policy on the simple PUMPdesign incurred more than incremental costs for several benchmarks dueto the large increase in miss handler runtime, but these are reducedwith the miss handler optimizations.

FIG. 13A (for CPI overhead) and FIG. 13B (for EPI overhead) illustratehow incremental addition of policies impacts runtime overhead by firstshowing the overhead of each single policy, then showing composites thatadd policies to Memory Safety, the most complex single policy. Theprogression makes it clearer what overhead comes simply from adding anypolicy as opposed to adding a higher-overhead policy. To get a sense ofscaling beyond the four policies here, the CFI policy (returns andcomputed-jumps/calls) and the taint tracking policy (code tainting andI/O tainting) are each broken into two parts. It is shown that theruntime overhead of additional policies tracks incrementally above thefirst complex policy (Memory Safety), with no appreciable runtime impacton the non-outliers (worst-case non-outlier rises from 9% to 10%overhead) and a larger increase (20-40%) in the two outliers as each newkind of policy is added due mostly to increased miss-handler resolutioncomplexity. Energy follows a similar trend with modest impact (geomeanrises from 60% to 70%) on the non-outlier policies, which account foreverything except GemsFDTD.

A brief summary of related work is identified in Table 2 reproduced atFIG. 15.

In accordance with a policy programming model in accordance withtechniques herein, a PUMP policy includes a set of tag values togetherwith a collection of rules that manipulate these tags to implement somedesired tag propogation and enforcement mechanism. Rules come in twoforms: the software layer (symbolic rules) or hardware layer (concreterules) of the system.

For example, to illustrate the operation of the PUMP, consider a simpleexample policy for restricting return points during program execution.The motivation for this policy comes from a class of attacks known asreturn-oriented programming (ROP), where the attacker identifies a setof “gadgets” in the binary executable of the program under attack anduses these to assemble complex malicious behaviors by constructingappropriate sequences of stack frames, each containing a return addresspointing to some gadget; a buffer overflow or other vulnerability isthen exploited to overwrite the top of the stack with the desiredsequence, causing the snippets to be executed in order. One simple wayof limiting ROP attacks is to constrain the targets of returninstructions to well-defined return points. This is accomplished byusing the PUMP by tagging instructions that are valid return points witha metadata tag target. Each time a return instruction is executed, themetadata tag on the PC is set to check to indicate that a return hasjust occurred. On the next instruction, the PC tag is check, verify thatthe tag on the current instruction is target, and signal a securityviolation if not. By making the metadata richer, it is possible toprecisely control which return instructions can return to which returnpoints. By making it yet richer, full CFI checking may be implemented.

From the point of view of the policy designer and the software parts ofthe PUMP 10, policies may be compactly described using symbolic ruleswritten in a tiny domain-specific language. An exemplary symbolic ruleand its program language is described, for example, in an APPENDIXattached hereto, entitled “PROGRAMMING THE PUMP, Hardware-AssistedMicro-Policies for Security”.

Symbolic rules may compactly encode a great variety of metadata trackingmechanisms. At the hardware level, however, a rule is needed forrepresentation that is tuned for efficient interpretation to avoidslowing down the primary computation. To this end, a lower level ruleformat, called concrete rules, may be introduced. Intuitively, eachsymbolic rule for a given policy can be expanded into an equivalent setof concrete rules. However, since a single symbolic rule might ingeneral generate an unbounded number of concrete rules, this elaborationis performed lazily, generating concrete rules as needed while thesystem executes.

For policies with metadata tags (e.g., which are richer than ROP), thetranslation from symbolic to concrete rules follows the same generallines, but the details become a bit more intricate. For example, thetaint-tracking policy takes tags to be pointers to memory datastructures, each describing an arbitrarily sized set of taints(representing data sources or system components that may havecontributed to a given piece of data). The symbolic rule for the loadopgroup says that the taint on the loaded value should be the union ofthe taints on the instruction itself, the target address for the load,and the memory at that address. The symbolic rule and its programlanguage is incorporated by reference from and is available for publicinspection in the paper entitled “PROGRAMMING THE PUMP,Hardware-Assisted Micro-Policies for Security” which was previouslyidentified.

To reduce the number of distinct tags (and, hence, pressure on the rulecache), metadata structures may be internally stored in canonical formand since tags are immutable, sharing is fully exploited (e.g., setelements are given a canonical order so that sets can be compactlyrepresented sharing common prefix subsets). When no longer needed, thesestructures can be reclaimed (e.g., by garbage collection).

An embodiment may utilize composite policies. Multiple orthogonalpolicies may be simultaneously enforced by letting tags be pointers totuples of tags from several component policies. (In general, multiplepolicies may not be orthogonal) For example, to compose the first returnopgroup (ROP) policy with the taint-tracking policy, let each tag be apointer to a representation of a tuple (r; t), where r is an ROP-tag (acode location identifier) and t is a taint tag (a pointer to a set oftaints). The cache lookup process is exactly the same, but when a missoccurs the miss handler extracts the components of the tuple anddispatches to routines that evaluate both sets of symbolic rules. Theoperation is allowed only if both policies have a rule that applies; inthis case the resulting tag is a pointer to a pair containing theresults from the two sub-policies.

In connection policy system and protection, the policy system exists asa separate region of memory within each user process. The policy systemmay include, for example, the code for the miss handler, the policyrules, and the data structures representing the policy's metadata tags.Placing the policy system in the process is minimally invasive with theexisting Unix process model and facilitates lightweight switchingbetween the policy system and the user code. The policy system isisolated from user code using mechanisms described next.

Clearly, the protection offered by the PUMP would be useless if theattacker could rewrite metadata tags or change their interpretation. Thetechniques described herein are designed to prevent such attacks. Thekernel, loader, and (for some policies) compiler is trusted. Inparticular, the compiler is relied on to assign initial tags to wordsand, where needed, communicate rules to the policy system. The loaderwill preserve the tags provided by the compiler, and that the path fromthe compiler to the loader is protected from tampering, e.g., usingcryptographic signatures.

An embodiment in accordance with techniques herein may use a standardUnix-style kernel which sets up the initial memory image for eachprocess. (It may be possible to use micro-policies to eliminate some ofthese assumptions, further reducing the size of the TCB). It is furtherassumed that, in such embodiments, the rule-cache-miss-handling softwareis correctly implemented. This is small, hence a good target for formalverification. One concern is to prevent user code running in a processfrom undermining the protection provided by the process's policy. Usercode should not be able to (i) manipulate tags directly—all tag changesshould be performed in accordance with the policy/policies rulescurrently in effect; (ii) manipulate the data structures and code usedby the miss handler; (iii) directly insert rules in the hardware rulecache.

In connection with addressing, to prevent direct manipulation of tags byuser code, the tags attached to every 64b word are not, themselves,separately addressable. In particular, it is not possible to specify anaddress that corresponds only to a tag or a portion of a tag in order toread or write it. All user accessible instructions operate on (data,tag) pairs as atomic units—the standard ALU operating on the valueportion and the PUMP operating on the tag portion.

In connection with the miss handler architecture in an embodiment inaccordance with techniques herein, the policy system may only beactivated on misses to the PUMP cache. To provide isolation between thepolicy system and user code, a miss-handler operational mode is added tothe processor. The integer register file is expanded with 16 additionalregisters that are available only to the miss handler, to avoid savingand restoring registers. Note, the use of 16 additional registers isillustrative and in practice may need to expand the integer registerfile to less/more registers. The PC of the faulting instruction, therule inputs (opgroup and tags), and the rule outputs appear as registerswhile in miss handler mode. A miss-handler-return instruction is added,which finishes installing a concrete rule into the cache and returns touser code.

In an embodiment in accordance with techniques herein, the normalbehavior of the PUMP 10 is disengaged while the processor 12 is inmiss-handler mode. Instead, a single hardwired rule is applied: allinstructions and data touched by the miss handler must be tagged with apredefined miss-handler tag that is distinct from the tags used by anypolicy. This ensures isolation between miss handler code and data andthe user code in the same address space. User code cannot touch orexecute policy system data or code, and the miss handler cannotaccidentally touch user data and code. The miss-handler-returninstruction can only be issued in miss-handler mode, preventing usercode from inserting any rules into the PUMP.

While previous work has used clever schemes to compactly represent orapproximate safety and security policies, this is often a compromise onthe intended policy, and it may trade complexity for compactness. Asdescribed herein, it is possible to include richer metadata thatcaptures the needs of the security policies both more completely andmore naturally with little or no additional runtime overhead. Ratherthan imposing a fixed bound on the metadata representation and policycomplexity, the PUMP 10 provides a graceful degradation in performance.This allows policies to use more data where needed without impacting thecommon case performance and size. It further allows the incrementalrefinement and performance tuning of policies, since even complexpolicies can easily be represented and executed.

With evidence mounting for the value of metadata-based policyenforcement, the present disclosure defines an architecture forsoftware-defined metadata processing and identifies accelerators toremove most of the runtime overhead. An architecture is introduced anddescribed herein with no bounds (i.e., free from any bound) on thenumber of metadata bits or the number of policies simultaneouslysupported along with four microarchitecture optimizations (opgroups, tagcompression, tag translation, and miss handler acceleration) thatachieve performance comparable to dedicated, hardware metadatapropagation solutions. The software defined metadata policy model andits acceleration will be applicable to a large range of policies beyondthose illustrated here, including sound information-flow control,fine-grained access control, integrity, synchronization, race detection,debugging, application-specific policies, and controlled generation andexecution of dynamic code.

Some non-limiting advantages of the various aspects and embodimentsdescribed herein provide (i) a programming model and supportinginterface model for compactly and precisely describing policiessupported by this architecture; (ii) detailed examples of policyencoding and composition using four diverse classes of well-studiedpolicies; and (iii) quantification of the requirements, complexity, andperformance for these policies.

The programming model of an embodiment as described herein may encode ahost of other policies. Information-flow control is richer than thesimple taint tracking models here, but tracking implicit flows can besupported either with RIFLE-style binary translation or by using the PCtag along with some support from the compiler. Micro-policies cansupport lightweight access control and compartmentalization. Tags can beused to distinguish unforgeable resources. Unique, generated tokens canact as keys for sealing and endorsing data, which in turn can be usedfor strong abstraction—guaranteeing that data is only created anddestructured by authorized code components. Micropolicy rules canenforce data invariants such as immutability and linearity.Micro-policies can support parallelism as out-of-band metadata forsynchronization primitives such as full/empty bits for data or futuresor as state to detect race conditions on locks. A system architect canapply specific micro-policies to existing code without auditing orrewriting every line.

The PUMP 10 design as described herein offers an attractive combinationof flexibility and performance, supporting a diverse collection oflow-level, fine-grained security policies with single policy performancecomparable to dedicated mechanisms in many cases while supporting richerand composite policies with mostly graceful performance degradation asrule complexity grows. Further, the mechanisms provided by the PUMP maybe used to protect its own software structures. An embodiment inaccordance with techniques herein may replace the special miss-handleroperational mode by implementing a “compartmentalization” micro-policyusing the PUMP 10 and using this to protect the miss-handler code.Finally, as described herein, orthogonal sets of policies may becombined, where the protections provided by each one are completelyindependent of the others. But policies often interact: for example, aninformation-flow policy may need to place tags on fresh regions beingallocated by a memory safety policy. Policy composition requiresanalysis in connection with both in expression and in efficient hardwaresupport.

What will now be described is a further example illustratingimplementation of a memory safety policy in an embodiment in accordancewith techniques herein that identifies all temporal and spatialviolations in heap-allocated memory. In at least one embodiment, foreach new allocation processing may be performed to make up a freshcolor-id, c, and write c as the tag on each memory location in the newlycreated memory block (e.g., such as via memset). The pointer to the newblock is also tagged c. Later, when processing is performed todereference a pointer, processing may include checking that thepointer's tag is the same as the tag on the memory cell to which thepointer references or points. When a block is freed, the tags on allcells of the block may be modified to a constant F representing freememory. The heap may be initially tagged F. A special tag, ⊥, may beused for non-pointers. Thus, generally, an embodiment may write a tag tfor a memory location that is either a color c or ⊥.

Because memory cells may contain pointers, in general each word inmemory may be associated with two tags. In such an embodiment, the tagon each memory cell be a pointer to a pair (c, t), where c is the id ofthe memory block in which this cell was allocated and t is the tag onthe word stored in the cell. An embodiment may use domain-specificlanguage based on the rule function described elsewhere herein forspecifying a policy in terms of symbolic rules. The rules for load andstore take care of packing and unpacking these pairs, along withchecking that each memory access is valid (i.e., the accessed cell iswithin the block pointed to by this pointer):

-   -   load: (-,-, c₁, -, (c₂, t₂))        -   →(-, t₂) if c₁=c₂    -   store: (-,-, t₁, c₂, (c₃, t₃))        -   →(-, (c₃, t₁)) if c₂=c₃

The checking performed in the foregoing and other rules shows up asconditions under which the symbolic rule is valid (e.g., c₂=c₃ above inthe store rule). The “-” symbol indicates the don't care fields in therule.

Address arithmetic operations preserve the pointer tag:

-   -   add: (-,-, c, ⊥, -)→(-, c)        To maintain the invariant that tags on pointers can only        originate from allocation, operations that create data from        scratch (e.g., loading constants) set its tag to ⊥.

In an embodiment implementing the memory safety policy, operations suchas malloc and free may be accordingly modified, for example, to tagmemory regions using the tagged instructions and ephemeral rules (e.g.,which may be deleted from the cache once they are used). In connectionwith malloc, processing may generate a fresh tag for the pointer to anew region via an ephemeral rule. For example, the rule for move may bean ephemeral rule such as:

-   -   move: (-, t_(malloc), t, -,-)→¹ (-, t_(newtag))        The arrow with the superscript of 1 (e.g., →¹) may denote an        ephemeral rule. The newly tagged pointer may then be used to        write a zero to every work in the allocated region using a        special store rule:    -   store: (-, t_(mallocinit), t₁, c₂, F)→(-, (c₂, t₁))        prior to returning the tagged pointer. At a later point in time,        free may use a modified store instruction to retag the region as        unallocated:    -   store: (-, t_(freeinit), t₁, c₂, (c₃, t₄)) (-, F)        prior to returning the region to the free list.

In such an embodiment using the memory safety policy, opgroups may beused to describe the rule set as follows:

-   (1) nop, cbranch, ubranch, ijump, return: (-,-,-,-,-)→(-,-)-   (2) ar2sld: (-,-, ⊥, ⊥,-)→(-, ⊥)-   (3) ar2sld: (-,-, c, ⊥, -)→(-, c)-   (4) ar2sld: (-,-, ⊥, c, -)→(-, c)-   (5) ar2sld: (-, -, c, c, -)→(-, ⊥)-   (6) ar1sld: (-, -, t, -, -)→(-, t)-   (7) ar1ld, dcall, icall, flags: (-, -, -, -, -)→(-, ⊥)-   (8) load: (-,-, c₁, -, (c₂, t₂))→(-, t₂) if c₁=c₂-   (9) store: (-,-, t₁, c₂, (c₃, t₃))→(-, (c₃, t₁)) if c₂=c₃    Λc_(i)∉{t_(mallocinit), t_(freeinit))-   (10) store: (-, t_(mallocinit), t₁, c₂, F)→(→, (c₂, t₁))-   (11) store: (-, t_(freeinit), t₁, c₂, (c₃, t₄))→(→, F)-   (12) move: (-, t_(malloc), t, -,-,)→¹ (-, t_(newtag))-   (13) move: (-, t_(malloc), t, -,-,)→(-, t)

The symbolic rules used above for policy specification may be writtenusing variables, allowing a few symbolic rules to describe the policyover an unbounded universe of distinct values. The concrete rules storedin the rule cache, however, refer to specific, concrete tag values. Forexample, if 23 and 24 are valid memory block colors, an embodiment mayuse concrete rules with concrete instances of symbolic Rule (3) above inthe PUMP rule cache for c=23 and c=24. Assuming, for example, anembodiment encodes ⊥ as 0 and marks don't care fields as 0, the concreterules are for symbolic rule (3) above are:

-   -   ar2s1d: (0, 0, 23, 0, 0)→(0, 23)    -   ar2s1d: (0, 0, 24, 0, 0)→(0, 24)        Consistent with discussion elsewhere herein, in at least one        embodiment, the miss handler may obtain the concrete input tags        and execute code compiled from the symbolic rules to produce the        associated concrete output tags in order to insert rules into        the PUMP rule cache. When the symbolic rule identifies a        violation, control transfers to an error handler and no new        concrete rules are inserted into the PUMP rule cache.

What will now be described is an embodiment in accordance withtechniques herein based on the RISC-V architecture further extended withmetadata tags and the PUMP to support software defined metadataprocessing (SDMP) consistent with discussion herein. RISC-V may becharacterized as an open source implementation of reduced instructionset computing (RISC) instruction set architecture (ISA). In such anembodiment, metadata tags are placed on both instructions and data foreach word. In the RISC-V architecture, words are 64 bits. The RISC-Varchitecture provides different word size variants—RV64 with a word sizeof 64 bits and RV32 with a word size of 32 bits. The width or size ofthe registers and user address space may vary with the word size. Tagsize or width may be independent of word size or width but may moretypically be the same in an embodiment. As known in the art, the RISC-Varchitecture has 32 bit instructions and thus an embodiment supportingand operating using the 64 bit word size may store 2 instructions in asingle tagged word. The foregoing and other aspects of the RISC-Varchitecture are discussed elsewhere herein in connection with use ofdifferent techniques and features in connection with extending theRISC-V architecture for use with metadata tags, the PUMP and SDMP.

The RISC-V architecture includes user-level instructions as described,for example, in “The RISC-V Instruction Set Manual Vol. I, User-LevelISA, Version 2.0”, May 6, 2014, Waterman, Andrew, et. al., (alsoreferred to as the “RISC-V user level ISA”) which is incorporated byreference herein, and is publically available, for example, at theRISCV.ORG website, and through the University of California at Berkeleyas Technical Report UCB/EECS-2014-54. The RISC-V architecture alsoincorporates a privileged architecture including privileged instructionsand additional functionality needed for running operating systems,attached external devices, and the like, as described, for example, in“The RISC-V Instruction Set Manual Volume II: Privileged Architecture,Version 1.7”, May 9, 2015, also referred to as the “RISC-V privilegedISA”) which is incorporated by reference herein, and is publicallyavailable, for example, at the RISCV.ORG website, and through theUniversity of California at Berkeley as Technical ReportUCB/EECS-2015-49.

An embodiment of the RISC-V architecture may have four RISC-V privilegelevels as follows: level 0 for user/application (U) privilege level,level 1 for supervisor (S) privilege level, level 2 for hypervisor (H)privilege level, and level 3 for machine (M) privilege level. In theforegoing, RISC-V privilege levels may be ranked, highest to lowest,from 0 to 3 where level 0 denotes the highest or greatest level ofprivilege and level 3 denotes the lowest or minimum privilege level.Such privilege levels may be used to provide protection betweendifferent components and attempts to execute code that performoperations not permitted by the current privilege level or mode willcause an exception to be raised such as traps into an underlyingexecution environment. Machine level has the highest privileges and isthe only mandatory privilege level for a RISC-V hardware platform. Coderun in machine-mode (M-mode) is inherently trusted, as it has low-levelaccess to the machine implementation. User-mode (U-mode) andsupervisor-mode (S-mode) are intended for conventional application andoperating system usage respectively, while hypervisor-mode (H-mode) isintended to support virtual machine monitors. Each privilege level has acore set of privileged ISA extensions with optional extensions andvariants. It should be noted that an implementation of the RISC-Varchitecture must support at least the M-mode and most implementationssupport at least U-mode and M-mode. S-mode may be added to providefurther isolation between code of supervisor-level operating system andother more privileged code executing in M-mode. User or application codemay typically execute in U-mode until a trap (e.g., supervisor call,page fault) or interrupt occurs forcing a transfer of control to a traphandler which runs at one of the supported higher privilege modes orlevels (e.g., H, S or M mode). Code of the trap handler is then executedand control may then be returned to the original user code orapplication which caused the trap. Such execution of the user code orapplication may resume at or after the original trapped instruction inU-mode that triggered the trap handler invocation. Various combinationsof supported modes in a RISC-V implementation may include only: thesingle M mode, two modes—M and U, three modes—M, S and U, or all fourmodes M, H, S, U. In at least one embodiment described herein, all 4 ofthe foregoing privilege levels may be supported. At a minimum, anembodiment in accordance with techniques herein may support M AND Umodes.

The RISC-V architecture has control status registers (CSRs) that may beatomically read and modified by one or more associated privilege levels.Generally, a CSR may be accessible at a first of the four privilegelevels and any other of the four privilege levels higher than the first.For example, assume a program is executing in U-mode (level 3) and atrap, such as a rule cache miss, occurs whereby control is transferredto a trap handler, such as the rule cache miss handler code, running ata higher privilege or mode (e.g., any of levels 0-2). Upon theoccurrence of the trap, information may be placed in CSRs accessible tothe trap handler executing in M-mode, for example, that are nototherwise accessible to any other code executing at a lower privilegelevel (e.g., not accessible to code in H, S or U mode). In at least oneembodiment, the rule cache miss handler may run at a privilege levelabove the level of PUMP protection (e.g., may run in H-mode, S-mode, orM-mode). In such an embodiment, as described elsewhere herein, the tagdefinitions and policies may be global across an operating system (e.g.,per virtual machine) at the rule cache miss handler level whereby thesame tag definitions and policies may be applied across all executingcode. In at least one embodiment, per application or process policiesmay be supported where such policies are installed globally and the PC(program counter identifying the current instruction) and/or code may betagged to distinguish process or application-specific rules. In anembodiment where virtual machines (VMs) do not share memory, policiesmay be defined at a per-VM basis.

Consistent with discussion elsewhere herein, the PUMP may becharacterized as a rule cache for SDMP. There may be a mapping between aset of tags on the instruction and instruction inputs and tags for theresult of the operation. Tag processing is independent and parallel fromthe normal operations of the instruction. In at least one embodiment,the PUMP runs in parallel with the normal RISC-V operations, supplyingthe tags for the results of the operation. Since the PUMP is a cache,rule cache misses occur the first time the PUMP receives a particularinstruction, and thus a particular corresponding set of PUMP inputs(e.g., compulsory) or when the PUMP was unable to retain a rule in cache(e.g., capacity of cache exceeded therefore the rule was evicted fromthe rule cache, or perhaps conflict). Rule cache misses cause a misstrap that is then handled by code of a miss handler system (e.g., rulecache miss handler). Inputs may be communicated to the miss handlerthrough PUMP CSRs and rule insertion may be provided back to the PUMPalso through CSRs. This is discussed in more detail below. A firstembodiment is discussed elsewhere herein where there are 5 PUMP inputtags. As a variation, an embodiment may include a different number oftags and other PUMP inputs. The particular number of PUMP tag inputs mayvary with the instruction set and operands. For example, the followingmay be included as PUMP inputs in one embodiment based on the RISC-Varchitecture:

1. Opgrp—denotes particular opgroup include a current instruction.Generally, an opgroup is an abstraction of a group of instructions andis discussed elsewhere herein.

2. PCtag—tag on the PC

3. Cltag—tag on the instruction

4. OP1tag—tag on RS1 input to instruction

5. OP2tag—tag on RS2 input to instruction (or tag on CSR when a CSRinstruction)

6. OP3tag—tag on RS3 input to instruction

7. Mtag—tag on memory input to instruction or memory target of aninstruction

8. funct12 (funct7)—extended opcode bits that occur in some instructionsas described elsewhere herein.

9. subinstr—when there are multiple instruction packed in a word, thisinput identifies which instruction in the word is the currentinstruction being operated upon by the PUMP.

The following may be included as PUMP outputs in one embodiment based onthe RISC-V architecture:

1. Rtag—tag on result: destination register, memory, or CSR

2. newPCtag—tag on the PC after this operation (e.g., sometimes referredto herein as PCnew tag).

Information may be communicated, for example, from user code executingin U-mode at the time of the trap occurrence to a trap handler, such asthe rule cache miss handler, executing in M-mode via CSRs. In a similarmanner, information may be communicated between the trap handler inM-mode when resuming program execution in U-mode via CSRs whereinformation in the CSRs may be placed in corresponding registersaccessible in U-mode. In this manner, there may be mapping between theCSRs at one privilege level and registers at other privilege levels. Forexample, in an embodiment in accordance with techniques herein, a CSRmay be defined that is accessible to the M-mode handler and PUMP where aparticular instruction operand tag is written to the CSR upon theoccurrence of a trap to communicate the tag to the PUMP and rule cachemiss handler as an input. In a similar manner, the CSR may be used tocommunicate information from the trap handler and/or PUMP (operating ata privilege level higher than U-mode) to other code executing in U-modesuch as when resuming program execution after a rule cache miss (e.g.,where rule cache miss occurs when a matching rule is not found in thePUMP rule cache for a current instruction). For example, a CSR may beused to output or propagate PUMP output tags for PCnew and RD.Additionally, CSRs may be defined where different actions may occurresponsive to writing to a particular CSR. For example, the rule cachemiss handler code may write/insert a new rule into the rule cache of thePUMP by writing to a particular CSR. The particular CSRs defined mayvary with embodiment.

Referring to FIG. 25, shown is an example of CSRs that may be definedand used in one embodiment in accordance with techniques herein. Thetable 900 includes a first column 902 with the CSR address inhexadecimal, a second column 904 of privilege, a third column 906denoting the CSR name, and a fourth column 908 with a description of theCSR. Each line of the table 900 may identify information for a differentdefined CSR. Different ones of the CSRs in 900 are also describedelsewhere herein in more detail in connection with additional featuresthat may be included in an embodiment.

Rows 901 a-c identify CSRs having special tag values used for taggingcode and/or instructions by the PUMP. In at least one embodiment, thesboottag CSR defined by entry 901 a may include a first initial orstarting tag value used in a system. The foregoing starting tag valuemay be referred to as a bootstrap tag value. In one aspect, thebootstrap tag value may be characterized as a “seed” from which allother tags may be derived or based on. Thus, the bootstrap tag may beused in one embodiment as a starting point for generating all othertags. In a manner similar to initial loading of a starting location ofbootstrap code in the operating system, hardware may be used toinitialize the CSR 901 a to the particular predefined tag value used asthe bootstrap tag. Once the bootstrap tag has been read as part ofbooting a system in accordance with techniques herein, the sboottag CSRmay be cleared. For example, a privileged portion of operating systemcode may include instructions which invoke rules performing initial tagpropagation using the bootstrap tag value. Use of the bootstrap tag andtag generation and propagation are further described elsewhere herein.Row 901 b identifies a CSR containing the tag value used for taggingdata from a public untrusted source as described elsewhere herein. ForRow 901 c identifies a CSR containing a default tag value that may beused as a default tag value when tagging data and/or instructions.

Rows 901 d and e, respectively, denote the address and data for writingto the opgroup/care table (e.g., also referred to elsewhere herein as amapping or translation table including opgroups and care/don't care bitsfor opcodes). Writing to the CSR denoted by row 901 e triggers a writeto the opgroup/care table. Row 901 f identifies a CSR that may bewritten to in order to flush the PUMP rule cache. Rows 901 g-901 midentify CSRs providing tag inputs for a current instruction to the PUMPand rule cache miss handler. Rows 901 j-m each denote a differentoperand tag for an operand of the current instruction being processedcausing the rule cache miss whereby an instruction may include up to 4such operands (with 3 of the 4 operands being registers (CSRs 901 j-1)and a 4^(th) operand being a memory location with a tag stored in theCSR denoted by row 901 m). Row 901 n identifies a CSR holding theextended opcode bits when the opcode of the current instruction uses theextended func12 field as described elsewhere herein. Row 9010 identifiesa CSR indicating which subinstruction in a word is the currentinstruction being referenced. As discussed elsewhere herein, a singletagged word may be 64 bits and each instruction may be 32 bits wherebytwo instructions may be included in a single tagged word. The CSRdenoted by row 9010 identifies which of the two instructions is beingprocessed by the PUMP. Rows 901 p-q identify CSRs including PUMP outputtags, respectively, of the new PC (e.g., new PC tag for the nextinstruction) and the RD (destination register, address for result of thecurrent instruction). Writing to the CSR denoted by 901 q causes a writeof the rule (e.g., matching a current instruction that triggered a PUMPrule cache miss) into the PUMP rule cache. Row 901 r identifies atagmode for PUMP operation. Tagmodes are described in more detailelsewhere herein.

In at least one embodiment, the one or more tables (e.g. opgroup/caretable) used to store opgroups and care/don't care bits may be populatedby writing to CSR sopgrpvalue denoted by 901 e where the contents of theforegoing CSR 901 e is written to the address stored in the sopgrpaddrCSR denoted by 901 d. A rule may be written or installed into the PUMPrule cache responsive to writing to the srtag CSR define by entry 901 q.The rule written is the rule specifying tag values matching the opcode(or more specifically the opgroup for the opcode) and tag values for thecurrent instruction as input to the PUMP via PUMP CSRs (e.g., based onPUMP CSR inputs 901 g-o).

To allow tagging and tag protection on CSR operations, the dataflowallows CSR tags to be inputs to, and outputs from the PUMP. Inaccordance with the RISC-V architecture, there are read and writeinstructions, respectively, to read from, and write to, CSRs. Inconnection with a CSR instruction with the PUMP, the R2tag input to thePUMP is the current CSR tag. The CSR read/write instructions (e.g.,csrrc, csrrci, csrrs, csrrsi, csrrw, csrrwi) write two outputs: (1) RD,and (2) the CSR referenced by the instruction. In this case, the PUMPoutput R tag (or RD tag of the destination) specifies the CSR tag outputby the PUMP and copying the CSRtag directly to the register destinationtag:

RDtag←CSRtag

-   -   C SRtag←Rtag

In connection with privilege denoted by column 904, CSR mtagmode,defined by row 901 r, is accessible for read/write by code executing atthe machine or M-mode level. The remaining CSRs defined by rows 901 a-qare accessible for read/write by code executing at least at thesupervisor or S-mode level. Thus, the privileges indicated in column 904for the various CSRs denote a minimum RISC-V privilege level ofexecuting code in order for the code to access the particular CSR. Anembodiment may assign different RISC-V privilege levels with CSRs usedin an embodiment than as illustrated in the example 900.

An embodiment in accordance with techniques herein may define multipletag modes affecting tag propagation performed by the PUMP. The currenttag mode is identified by the value at a current in point in time storedin the CSR mtagmode as defined by row 901 r. In at least one embodiment,tag mode may be used in combination with the RISC-V defined privileges(e.g., M, H, S and U modes described above) to define a CSR protectionmodel used in connection with the PUMP.

In order to allow the rule cache miss handler placement to beconfigurable, a protection model may be utilized that further extendsthe RISC-V privileges. Rather than defining PUMP CSR access entirely byprivilege level, CSR access may be further defined relative to thecurrent tag mode in combination with the RISC-V privilege levels. Thus,in at least one embodiment in accordance with techniques herein, whetherexecuting code is allowed to access a CSR may depend on the minimumRISC-V privilege level of the CSR, the current tag mode and the currentRISC-V privilege level of the executing code. Tag modes are discussedbelow in more detail.

Referring to FIG. 26, shown is an example of tag modes that may be usedin an embodiment in accordance with techniques herein. The table 910includes the following columns —912 mtagmode bit encoding, 914 operationand 916 tag results. Each row of table 910 denotes information for adifferent possible tag mode. When tag mode is 000 as denoted by 911 a,the PUMP is off and is not in use and does not generate any tag results.When tag mode is 010, the PUMP writes the default tag on all results(e.g., Rtag for destination or result register or memory location).

In connection with rows 911 c-f, denoted are different tag modes thatmay be specified for engaging or disengaging the PUMP for code executingat different RISC-V privilege levels. When the PUMP is engaged, the PUMPmay be characterized as active, enabled and providing protection whencode is executed whereby the rules of its policies are enforced duringcode execution. In contrast, when the PUMP is disengaged, the PUMP maybe characterized as inactive, disabled and not providing protection whencode is executed whereby rules of its policies are not enforced duringcode execution. When the PUMP is disengaged, tags may be propagatedusing one or more default tag propagation rules rather than have tagspropagated based on evaluation of a rule with tag values matching thoseof the current instruction. Whether the PUMP is engaged or disengagedmay vary with the particular assumed level of trust and desired level ofprotection attributed to code that executes at different RISC-Vprivilege levels.

In connection with tag modes 911 c-f, all PUMP CSRs of the example 900,except for the mtagmode CSR denoted by 901 r, may be accessible onlywhen the PUMP is disengaged. That is, PUMP CSRs of the example 900,except for the mtagmode CSR denoted by 901 r, are only accessible tocode executing at a current RISC-V operating privilege or mode that ismore privileged than the highest ranked PUMP privilege denoted by thetag mode (e.g., highest ranked privilege denoted by 911 c is U mode,highest ranked privilege denoted by 911 d is S mode, highest rankedprivilege denoted by 911 e is H mode, and highest ranked privilegedenoted by 911 f is M mode).

When tag mode is 100 as denoted by 911 c, the PUMP is disengaged and notoperational when the RISC-V privilege level denotes a higher or moreelevated privilege level than U-mode. Thus, tag mode 911 c indicatesthat the PUMP and its rules providing protection are only engaged andenforced when code executes at U-mode thereby indicating that codeexecuting at a privilege level higher than U-mode (e.g., at S, M, or Hmode) is trusted. When tag mode is 100 as denoted by 911 c and theRISC-V protection level of executing code is S, M, or H mode, the PUMPis disengaged and its CSRs are accessible to code that executes only inthe S, M or H mode (e.g., CSRs are not accessible to code executing inU-mode).

When tag mode is 101 as denoted by 911 d, the PUMP is disengaged and notoperational when the RISC-V privilege level denotes a higher or moreelevated privilege level than S-mode. Thus, tag mode 911 d indicatesthat the PUMP and its rules providing protection are only engaged andenforced when code executes at S-mode and U-mode thereby indicating thatcode executing at a privilege level higher than S-mode (e.g., at M or Hmode) is trusted. When tag mode is 101 as denoted by 911 d and theRISC-V protection level of executing code is M or H mode, the PUMP isdisengaged and its CSRs are accessible to code that executes only in theM or H mode (e.g., CSRs are not accessible to code executing in S or Umode).

When tag mode is 110 as denoted by 911 e, the PUMP is disengaged and notoperational when the RISC-V privilege level denotes a higher or moreelevated privilege level than H-mode. Thus, tag mode 911 e indicatesthat the PUMP and its rules providing protection are only engaged andenforced when code executes at H-mode, S-mode and U-mode therebyindicating that code executing at a privilege level higher than H-mode(e.g., at M mode) is trusted. When the tag mode is 110 as denoted by 911e and the RISC-V protection level of executing code is M mode, the PUMPis disengaged and its CSRs are accessible to code that executes only inthe M mode (e.g., CSRs are not accessible to code executing in U, H or Smode).

When tag mode is 111 as denoted by 911 f, the PUMP is always engaged andoperational for all the RISC-V privilege levels of M, H, S and U. Thus,tag mode 911 f indicates that the PUMP and its rules providingprotection are engaged and enforced when code executes at any of M-mode,H-mode, S-mode and U-mode thereby indicating that no code is inherentlytrusted. With tag mode=111 as denoted by 911 f, the PUMP is neverdisengaged and its CSRs are not accessible to any executing code.

In connection with tag modes denoted by rows 911 c-f, when the currentRISC-V privilege level of executing code is higher than the highestengaged PUMP level denoted by tag mode, the PUMP may be disengaged andtags may be propagated using one or more default tag propagation rules.

When the tag mode has an encoding of 000 as denoted by row 911 a(indicating the PUMP is off) or when the tag mode has an encoding of 010as denoted by row 911 b (indicating write default mode), all CSRs oftable 900 may only be accessible by code executing in M mode.

Thus, in at least one embodiment in accordance with techniques herein,whether executing code is allowed to access a CSR may depend on theminimum RISC-V privilege level of the CSR (such as specified in column904 of table 900), the current tag mode, and the current RISC-Vprivilege level of the executing code. For example, in the RISC-Varchitecture without considering tag mode, code executing at U-mode isnot allowed to access any of the CSRs defined in 900 due to the minimumprivilege levels denoted by 904 for all such CSRs. However, withoutconsidering tag mode, code executing with a privilege of at least H-modeis allowed access to all CSRs of 900 except for 901 r and code executingin M mode is allowed to access all CSRs of 900. Now consider determiningCSR access for CSRs of 900 in accordance with the minimum RISC-Vprivilege of 904 and tag mode. For example, consider code portion Aexecuting at H-level. Code portion A is allowed to access CSRs 901 a-q(of table 900) when the tag mode is 100 as denoted by 911 c or when thetag mode is 101 as denoted by 911 d. Code portion B executing in S mode,however, may not be allowed access to CSRs 901 a-q since it does nothave the minimum privilege level specified by the defined CSR privilegelevel in 904 for such CSRs. Thus, for example, code portion A may be thecache miss handler in one embodiment executing at H-level using CSRs asdefined in table 900. As a second example, assume the minimum RISC-Vprivilege defined for CSRs 901 a-q is SRW (denoting S mode as theminimum privilege level to access such CSRs). Code portion A executingin H mode is allowed to access CSRs 901 a-q when the tag mode is 100 asin 911 c and when the tag mode is 101 as in 911 d, and code portion Bexecuting in S mode is allowed to access CSRs 901 a-q when the tag modeis 100 as in 911 c. Thus, code portion A or B may be code of the cachemiss handler.

In at least one embodiment, the off tag mode of 911 a may be current tagmode when the PUMP is off such as during appropriate parts of the bootup process. The default tag tag mode of 911 b may be the current tagmode when initializing memory locations to have the same default tag(e.g., as denoted by CSR 901 c). Generally, although 4 privilege modesare specified in the RISC-V architecture, an embodiment mayalternatively use a different number of privilege modes where a firstprivilege level denotes a user mode or unprivileged mode and a secondprivilege level denotes an elevated or privileged mode of execution(e.g., similar to kernel mode in a UNIX based operating system). In suchan embodiment, the PUMP may be engaged and enforcing policy rules whenexecuting code in user or unprivileged mode and the PUMP may bedisengaged (e.g., PUMP protection off or not enforcing rules) whenexecuting code in the second elevated privilege mode. In this manner, anembodiment may disengage the PUMP when executing trusted or elevatedprivilege code such as a miss handler to store a new rule in the PUMPrule cache.

As noted above, an embodiment may use default propagation rules todetermine PUMP outputs new PC tag and R tag, for example, when the PUMPis disengaged and/or when the rule specifies don't care for the PUMPoutputs new PC tag and R tag (e.g., such don't care values may beindicated by the care vector for a particular opcode of the currentinstruction). In one embodiment, the following may denote logic embodiedin default propagation rules used.

-   -   newPCtag is PCtag for default propagation    -   Rtag is RS1tag for CSR read and write operations; RDtag is        assigned RS2tag (CSRtag)        -   allows tags to swap along with data values        -   RDtag←RS2tag←original CSRtag        -   CSRtag←Rtag←original RS1tag    -   Rtag is RS2tag (CSRtag) for CSRR?I, CSRRS, CSRRC        -   keeps CSRtag unchanged        -   RDtag←RS2tag←original CSRtag        -   CSRtag←Rtag←original RS2tag←original CSRtag    -   Rtag is PCtag for JAL and JALR instructions (this is for the        return address)    -   Rtag is PCtag for AUIPC instruction. In RISC-V, the AUIPC (add        upper immediate to PC) instruction is used to build PC-relative        addresses and uses the U-type format. AUIPC forms a 32-bit        offset from the 20-bit U-immediate, filling in the lowest 12        bits with zeros, adds this offset to the PC, then places the        result in register rd.    -   Rtag is Cltag for LUI instruction. In RISC-V, the LUI (load        upper immediate) instruction is used to build 32-bit constants        and uses the U-type format. LUI places the U-immediate value in        the top 20 bits of the destination register RD, filling in the        lowest 12 bits with zeros.    -   Rtag is RS1tag for non-memory, non-CSR, non-JAL(R)/AUIPC/LUI        operations    -   Rtag is RS2tag for memory write operations    -   Rtag is Mtag for memory load operations

In at least one embodiment of techniques herein based on the RISC-Varchitecture, a new PUMP miss trap may be defined for a rule cache missoccurrence. The PUMP miss trap may have a lower priority than virtualmemory fault or illegal instructions.

In at least one embodiment in accordance with techniques herein usingthe RISC-V architecture, strict separation and isolation between dataand metadata may be maintained where there is separation and isolationbetween tag metadata processing and normal instruction processing. Thus,separate execution domains between metadata rule processing and normalor typical program instruction execution may be maintained. Metadataprocessing performed using the PUMP for tags associated withinstructions and data of executing code may be performed. A PUMP rulecache miss results in a trap causing transfer of control to a rule cachemiss handler that generates or retrieves a rule matching the currentinstruction and stores the rule in the PUMP rule cache. Information maybe communicated between the above-noted execution domains using CSRs.When switching from the instruction execution domain of an executingprogram to the metadata rule processing domain (such as when the rulecache miss handler is triggered via rule cache miss trap), tags andother information relevant to the instruction (causing the trap) may beprovided as inputs to the PUMP and also the miss handler using CSRs. Ina similar manner, when transferring control from the metadata ruleprocessing domain to the instruction execution domain of an executingprogram (such as when returning from the rule cache miss handler afterhandling a rule cache miss trap), PUMP outputs may be communicated usingCSRs where the contents of the CSRs are then stored in correspondingmapped registers in the instruction execution domain. Consistent withdiscussion herein, an instruction which does not map to a rule (e.g., nomatching rule for the instruction is located in the cache and the cachemiss handler determines that no such matching rule exists for thecurrent instruction) indicates that the rule is not allowed to executewhereby a trap or other event is triggered. For example, the processormay stop execution of the current program code.

In this manner, there may be strict separation between the foregoingdomains and associated data paths even though the same RISC-V processorand memory may be used in both domains. Using techniques herein, noinstructions of executing code are allowed to read or write metadatatags or rules. All metadata transformations including tagginginstructions and data may be done through the PUMP. Similarly, ruleinsertion into the PUMP cache may be performed only by the rule cachemiss handler of the metadata subsystem. In connection with processingperformed by the metadata subsystem or processing system, the metadatatags of the executing code are placed in PUMP CSRs and become the “data”input to, and operated upon, by the metadata system (e.g., pointers areinto metadata memory space). The metadata subsystem reads the PUMPinputs via the PUMP input CSRs for processing in accordance with rules.If the instruction is allowed to proceed via the rules, the PUMP writestag results (e.g., such as for PC new and R tag) to defined PUMP outputCSRs. Rule insertion into the rule cache may be triggered responsive towriting to a particular CSR (e.g., such as the srtag CSR in 901 q). Inthis manner, all tag updates are done through rules in the PUMP andcontrolled by the metadata subsystem. Only the metadata subsystem caninsert rules into the PUMP cache via the cache miss handler invoked uponoccurrence of a rule cache miss. Additionally, in at least oneembodiment as described herein using the RISC-V architecture, theforegoing separation between metadata processing and normal instructionprocessing may be maintained without adding any new instructions beyondthose in the “RISC-V user level ISA” and the “RISC-V privileged ISA”.Consistent with discussion elsewhere herein, an embodiment in accordancewith techniques herein may maintain strict separation and isolationbetween data and metadata whereby there is separation between metadataprocessing based on tags and normal instruction processing. In at leastone embodiment, such separation may be maintained by having a separatephysical metadata processing subsystem with a separate processor and aseparate memory. Thus, a first processor and a first memory may be usedwhen processing instructions of an executing program and a secondprocessor and a second memory may be included in the metadata processingsubsystem for use with performing metadata processing such as whenexecuting code of the rule cache miss handler.

Referring to FIG. 27, shown is an example 1000 of components that may beincluded in an embodiment in accordance with techniques herein. Theexample 1000 includes a first subsystem or processor 1002 used inconnection with normal processing for an executing program and ametadata processing subsystem or processor 1004. The first subsystem1002 may be characterized as a program execution subsystem used inconnection with normal program execution. The subsystem 1002 is aprocessor that includes components used in connection with executingprogram code and using data where such code and data includes tags asdescribed elsewhere herein for use with the metadata processingsubsystem 1004. The subsystem 1002 includes memory 1008 a, instructionor I-store 1008 b, ALU (arithmetic and logic unit) 1008 d, and programcounter (PC) 1008 e. It should be noted that the PUMP 1003 may be usedin connection with execution of code in subsystem 1002 but may beconsidered as part of the metadata processing subsystem 1004. All codeand data in the subsystem 1002 may be tagged such as generallyillustrated by tag 1002 a associated with data 1002 b where 1002 a and1002 b may be stored in memory 1008 a. Similarly, element 1001 a denotesa tag on an instruction of the PC 1008 e, 1001 b denotes tags ofinstructions 1008 b, 1001 c denotes tags of memory locations 1008 a, and1001 d denotes tags of registers 1008 c.

Metadata processing subsystem 1004 is a processor (also referred to asthe metadata processor) that includes components used in connection withmetadata rule processing using tags of a current instruction andassociated data provided as inputs to the PUMP 1003. The PUMP 1003 maybe as described elsewhere herein and includes a rule cache. For example,in at least one embodiment, the PUMP 1003 may include the componentsillustrated in FIG. 22. More detailed illustration and example ofcomponents of the PUMP 1003, associated PUMP CSRs used for PUMP inputsand outputs and associated logic that may be included in at least oneembodiment in accordance with techniques herein are described in moredetail below and elsewhere herein. The subsystem 1004 is a separateprocessor used for metadata processing and includes components similarto those of subsystem 1002. The subsystem 1004 includes memory 1006 a,I-store 1006 b, register file 1006 b, and ALU 1006 d. Memory 1006 a mayinclude metadata structures used in connection with metadata ruleprocessing. For example, memory 1006 a may include the structures ordata that is pointed to by a tag that is a pointer. Examples of apointer tag and structures/data pointed to by the pointer tag aredescribed elsewhere herein such as in connection with a CFI policy.I-store 1006 b and memory 1006 a may include instructions or code suchas the miss handler that performs metadata processing. The metadataprocessor 1004 does not need access to other components of 1002, such asdata memory 1008 a used in connection with program execution, since themetadata processor 1004 only performs metadata processing (e.g., basedon tags and rules). The subsystem 1004 includes its own components, suchas a separate memory 1006 a, and does not need to store metadataprocessing code and data in the subsystem 1002. Rather, any information,such as tags of a current instruction that may be used by the PUMP 1003are provided as inputs (e.g., PUMP inputs 1007) to the metadataprocessing subsystem 1004.

The example 1000 illustrates an alternative embodiment having a separatemetadata processing subsystem 1004 rather than performing metadataprocessing on the same subsystem as used for normal program execution asdescribed elsewhere herein. For example, rather than have a separatemetadata processor or subsystem 1004, an embodiment may include only thePUMP 1003 and subsystem 1002. In such an embodiment with a singleprocessor, CSRs may be used as described herein to communicateinformation between the metadata processing and normal processing modeexecuting a user program to thereby provide isolation and separation. Insuch an embodiment with a single processor rather than a separatemetadata processor, code of the miss handler may be stored in the singlememory in a manner so that it is protected. For example, without aseparate metadata processor or subsystem, the code of the miss handlermay be protected using tags as described elsewhere herein to limitaccess, may be mapped to a portion of memory that is not addressable byuser code, and the like.

What will now be described are further details regarding PUMP I/O(input/output). It should be noted that PUMP I/O described below appliesto embodiments of the PUMP which may use the same processor or subsystemas for normal code execution as well as those which may use separateprocessors or subsystems such as in the example 1000. Furthermore, thePUMP I/O described below may be used with an embodiment based on theRISC-V architecture and may be generalized for use with other processorarchitectures.

Referring to FIG. 28, shown is an example 1010 summarizing PUMP I/O inan embodiment in accordance with techniques herein. As describedelsewhere herein such as in connection with FIGS. 1 and 24, the PUMPoperates in stages 5 and 6. The PUMP inputs are used in connection withnormal PUMP verification (e.g., verify whether current instruction isallowed using policy rules) to find a matching rule, if any, in the rulecache of the PUMP for the current instruction. The normal PUMPverification may occur for every instruction such as part at stage 5 asdescribed elsewhere herein with a 6 stage pipeline. Additionally, thePUMP inputs may be used in connection with controlling rule insertioninto the rule cache such as may occur in stage 6 of the 6 stagepipeline. The PUMP I/O associated with normal PUMP verification isdenoted in the example 1010 by inputs and outputs in the verticaldirection from top (inputs 1012) to bottom (outputs 1014). The PUMP I/Oassociated with controlling rule insertion into the PUMP rule cache isdenoted in the example 1010 by inputs and outputs in the horizontaldirection from left (inputs 1016) to right (output 1018) Additionally,element 1012 denotes additional inputs also used in connection with ruleinsertion, as described elsewhere in more detail.

First, consider the PUMP I/O associated with normal PUMP verificationprocessing. PUMP inputs 1012 may include tags, such as the PC tag, theCI tag, instruction operand tags (e.g., OP1 tag, OP2 tag or CSR tag (forCSR-based instructions in RISC-V), OP3 tag, M Tag (for a memory locationtag for memory instructions. Note that Mtag may also be referred toherein as the MR tag for a memory instruction), opcode information(e.g., op group denoted by Opgrp input, funct12 (funct7) input forRISC-V for extended opcodes, subinstr input providing an indicator ofwhich instruction is the current instruction in an instruction wordincluding multiple instructions such as in examples 200 and 220) andcare input bits. The Opgrp may be the opgroup for the currentinstruction where Opgrp may be an output of a prior stage (e.g., stage 3or stage 4) as described elsewhere herein. Funct 12 (funct7) PUMP inputmay be the additional opcode bits, if any, for those RISC-V opcodesusing additional bits of the instruction word (e.g., example 400). PUMPoutputs 1014 may include Rtag (e.g., tag for the instruction resultregister or destination memory location), PC new tag (denoting thepropagated tag placed on the PC used for the next instruction), and anindicator 1014 a denoting whether there has been a PUMP rule cache missresulting in a trap to the miss handler in stage 6.

The care bits 1012 a may denote which PUMP inputs 1012 and which PUMPoutputs 1014 are cared/not care about (e.g., ignored) for a particularinstruction. Care bits regarding PUMP inputs may include a care bit forfunct12 and a second care bit for funct7. As described elsewhere herein,both of the foregoing care bits denote whether the particular opcode ofthe current instruction includes any bits for the extended 12 opcode bitportion for a RISC-V instruction (e.g. 404 a of the example 400). Ifboth funct12 and funct7 care bits are “don't care”, then all 12 bits ofthe extended 12 opcode bit portion are masked out. If funct7 indicates“care”, then all the bottom 5 bits of the extended 12 opcode bit portionare masked out. If funct12 indicates “care”, then there is no masking ofthe extended 12 opcode bit portion.

Now consider the PUMP I/O associated with controlling rule insertioninto the PUMP rule cache. PUMP inputs 1016 may be used in combinationwith inputs 1012 in connection with the PUMP cache rule insertion. PUMPinputs 1016 may include Opldata (an output from the metadata processoror subsystem), the instruction (from stage 6), and the tag mode (anoutput from the metadata processor or subsystem) and privilege (privdenoting the RISC-V privilege). The tag mode and priv inputs of 1016 areused by the metadata processor or subsystem to determine whether code,such as the miss handler or other code, executing in the metadataprocessor has sufficient privilege to access the CSRs described belowand elsewhere herein providing the various inputs to the metadataprocessor (e.g., such as inputs 1012). Rdata 1018 is an input to themetadata processor or subsystem for use in stage 6 (e.g., cache misshandler processing input). It should be noted that Op1data, R data, andother items of the example 1010 are described in more detail infollowing paragraphs and figures.

Thus generally, in the example 1010, element 1012 denotes inputs to thePUMP and metadata processor from the processor executing user code (e.g.non-metadata processor or subsystem such as 1002), element 1014 denotesoutputs generated by the metadata processor, element 1016 denotesoutputs generated by the metadata processor input to the PUMP, andelement 1018 denotes an input to the metadata processor.

Referring to FIG. 29, shown is an example 1020 summarizing the I/O inconnection with the opgroup/care table (e.g., element 422 of example420) in an embodiment in accordance with techniques herein. As describedelsewhere herein, the opgroup/care table may be used for eachinstruction to lookup and output an opgroup and care bits for the opcodeof the current instruction. This first flow of I/O is illustrated in1020 by inputs and outputs in the vertical direction from top (input1022) to bottom (outputs 1024). As described elsewhere herein, the input1022 may be the opcode or a portion thereof (e.g., such as described inconnection with example of the opcode portion in the example 420) usedan index into the opcode/care table. Input 1022 may be from stage 3. Theoutputs 1024 may be the opgroup (opgrp) and care bits for the particularopcode. The outputs 1024 are inputs to stage 5 (e.g., two of the PUMPinputs oprgrp and care as included in 1012).

A second flow of I/O is illustrated in 1020 by inputs and outputs in thehorizontal direction from left (inputs 1026) to right (output 1028). Thesecond flow of I/O in 1020 is illustrative of processing performed inconnection with controlling the selection of PUMP output Rdata 1028which is input to the metadata processor or stage 6. The inputs 1026 areas described above in connection with 1016. The output 1028 is asdescribed above in connection with 1018.

Referring to FIG. 30, shown is an example 1030 abstractly representingprocessing performed by the PUMP in an embodiment in accordance withtechniques herein. The example 1030 includes PUMP control 1031 whichcorresponds to the PUMP control for rule insertion described above inconnection with the horizontal PUMP I/O flow in the example 1010 (e.g.,elements 1012, 1016 and 1018). The example 1030 includes masking 1032,hash 1034, rule cache lookup 1036 and output tag selection 1038 whichcorresponds to the normal PUMP verification path I/O flow performed foreach instruction as described above in connection with the vertical PUMPI/O flow in the example 1010 (e.g., elements 1012 and 1014). The masking1032 denotes applying the care bits of 1012 to mask out unused PUMPinputs of 1012. The hash 1034 denotes computation of the hash usedduring the rule cache lookup denoted by 1036. Components that may beused in implementing the logic denoted by 1032, 1034 and 1036 in oneembodiment are illustrated and described in connection with FIG. 22. Theoutput tag select 1038 denotes selection of PUMP outputs Rtag and PC newtag as included in 1014 based on the care vector bits (care included ininputs 1012) and the htagmode CSR (denoting the current tag mode).

Referring to FIG. 31, shown is an example 1040 denoting components thatmay be used to implement logic of the output tag select 1038 of the PUMPin an embodiment in accordance with techniques herein. The example 1040includes multiplexers (MUXs) 1043 a-b. Generally, MUX 1043 a may be usedto select the final tag value for PC new tag 1043 as output by the PUMP(e.g., PC new tag of 1014), and MUX 1043 b may be used to select thefinal tag value for R tag 1047 as output by the PUMP (e.g., R tag of1014). Element 1042 denotes the inputs used as the selector for MUX 1043a. The inputs 1042 are used are used to select either 1041 a or 1041 bas the PC new tag 1043. The inputs 1042 may include the PCnew tag carebit (e.g., from care bits of 1012) logically ANDed (&&) with engaged (aBoolean denoting whether or not the PUMP is engaged). Element 1043denotes the inputs used as the selector for MUX 1043 b. The inputs 1043are used to select one of the inputs denoted by 1045 a-1045 b as the Rtag 1047. The inputs 1043 may include the Rtag care bit (e.g., form carebits of 1012) logically ANDed (&&) with engaged. Thus, generally thecare bits included in the PUMP inputs 1012 identify which PUMP inputsare don't cares (are masked out) and which PUMP outputs (Rtag and PCnewtag) are don't cares (are masked out). Also, outputs 1043 and 1047 aretreated as “don't care” values when the PUMP is disengaged because theprocessor is running at a higher privilege level than the currenttagmode specifies as the threshold for PUMP operation.

Element 1049 denotes how the Boolean engaged is determined as a functionof current RISC-V privilege and the current tagmode. Element 1049includes a logical expression using standard notation known in the artwhereby “A==B” denotes if a logical test for equality between A and B,“A && B” denotes a logical AND operation of A and B, and “A∥B” denotes alogical inclusive OR operation between A and B.

Element 1041 a and 1045 a denote inputs to 1043 a which are outputs fromthe rule cache lookup 1036. PC tag 1041 b is the PC tag included in thePUMP inputs 1012. Other inputs 1041 b generally denote multiple otherinputs that may be possibly selected as the final R tag 1047 output bythe PUMP. For example, in one embodiment, other inputs 1041 b mayinclude M tag, PC tag, CI tag, OP1 tag, OP2 tag, OP3 tag, and possiblyothers depending on the instruction. The particular R tag output 1047may vary with the particular RISC-V instruction/opcode.

The following may summarize particular values for R tag 1047 and PC newtag 1043 generated as PUMP output values in one embodiment. It should benoted that following indicates particular R tag output values fordifferent RISC-V instructions. Thus, the particular R tag values outputas the final PUMP R tag value may vary with the instructions thatutilize such PUMP outputs in connection with subsequent metadataprocessing.

1. PCtag does not change when output care bit is off for PC new tag

2. Rtag is Op1tag for CSRRW operations

3. Rtag is Op2tag (CSRtag) for CSRR?I, CSRRS, CSRRC operations

4. Rtag is PCtag for JAL and JALR instructions

5. Rtag is PCtag for AUIPC instruction

6. Rtag is CItag for LUI instruction

7. Rtag is Op1tag for non-memory, non-CSR, non-JAL(R)/AUIPC/LUIoperations when output care bit is off (indicates care for Rtag).

8. Rtag is Op2tag for memory write operations when output care bit isoff

9. Rtag is Mtag for memory load operations when output care bit is off

Referring to FIG. 32, shown is an example 1050 of components that may beused to control PUMP I/O in an embodiment in accordance with techniquesherein. Generally, referring back to the example 1030, the components of1050 may comprise another layer logically on top of 1032 (e.g.,interfacing with components of FIG. 22). Elements M1-M14 denotemultiplexers used for selection of various inputs thereto. Element 1052generally denotes the inputs opcode, PC tag, CI tag, Op1 tag, Op2 tag,Op3 tag, and M tag from 1012 for the current instruction. Element 1056generally refers to a row of registers used to store the selectedoutputs of multiplexers M1-M7. In one embodiment based on the RISC-Varchitecture, each of the boxes in row 1056 may be a register, and inparticular, a CSR containing a particular value as described elsewhereherein (e.g. the example 900 denoting CSRs that may be used in oneembodiment).

It should be noted that element 1052 of the example 1050 does notinclude all of the inputs of 1012. For example, funct12 (funct7) andsubinstr inputs of 1012 are not illustrated in the example 1050 forpurposes of simplicity. However, one of ordinary skill in the artappreciates that the inputs funct12 (funct7) and subinstr from 1012 mayalso be included in 1052. More generally, the inputs 1052 may be adaptedfor the particular inputs to metadata rule processing that may be usedin an embodiment.

When the PUMP is performing processing for an instruction for normalPUMP verification (e.g., verify whether current instruction is allowedusing policy rules), the inputs 1052 may simply pass through as theoutputs 1054. The outputs 1054 in this case flow through to the PUMP asinputs such as inputs to components of FIG. 22 (or more generally passthrough to the metadata processor or subsystem) for metadata processing.With normal PUMP verification, the PUMP may then produce outputs 1014(e.g., Rtag and PC new tag if a matching rule for the currentinstruction is found in the rule cache and otherwise generating a cachemiss 1014 a).

Upon the occurrence of a rule cache miss, as a first step, the currentvalues from 1052 for the current instruction are loaded into registersG1-G7 of 1056. Thus, G1-G7 includes a snapshot of the opcode and tagvalues for the current instruction that caused the rule cache miss andsuch values may now be used in connection with subsequent processing bythe cache miss handler reading out the one or more desired values ofG1-G7 as needed for such processing.

Thus, in a second step, the cache miss handler executes, reads as inputsvalues from G1-G7 and generates the new rule for the currentinstruction. Multiplexer M16 may be used to control selection of thevarious possible inputs from G1-G7 where the selected output from M10 isdenoted as R data 1053 for processing by the cache miss handler (e.g.which may execute on either the same processor as when executing programcode or may otherwise executed on a separate metadata processor such asin the example 1000). Given the inputs G1-G7 for the current instructioncausing the rules cache miss, the cache miss handler performs processingto determine the new rule to be inserted into the cache. The cache misshandler generates outputs R tag and PC new tag for the new rule justdetermined, writes the Rtag to the Rtag CSR G8, and writes the PC newtag to the PC new CSR G9. In the example 1050, Op1data 1051 denotes theoutputs generated by the metadata processor such as the outputs Rtag andPC new tag for the new rule where such outputs are then stored in CSRsG8 and G9 as described.

At this time, the values in CSRs G1-G9 are the tag values for the newrule just generated by the cache miss handler and may beinserted/written to the rule cache as the new rule in a third step. Inat least one embodiment using techniques herein with the RISC-Varchitecture, writing to the R tag CSR denoted by G8 triggers writing ofthe new rule (e.g., contents of CSRs G1-G9) to the rule cache. Inconnection with rule insertion, CSRs G1-G7 are provided as output 1052and CSRs G8 and G9 are provided as output 1055 to the PUMP for storinginto the rule cache. More specifically, in one embodiment, the outputs1052 and 1055 may be provided to the components of FIG. 22 for ruleinsertion.

In the simple case, an embodiment may insert one new rule to satisfy thecurrent rule miss by writing the contents of CSRs G1-G9 for the new ruleto the PUMP rule cache as just described (e.g., via outputs 1052 and1055). In such an embodiment, the multiplexers M1-M7 are not neededsince Op1data 1051 output by the metadata rule processor executing thecache miss handler only generates R tag and PC new tag for the new rule.However, an embodiment may also allow for rule prefetching or insertingmultiple rules into the rule cache. For example, upon the occurrence ofa rule cache miss, the cache miss handler may determine multiple rulesto be written to/inserted into the rule cache rather than just a singlenew rule for the current instruction. In this case, the Op1data 1051 mayinclude additional new values for the opcode, PCtag, CItag, Op1tag,Op2tag, Op3tag and Mtag (written to CSRs G1-G7) as well as new valuesfor the Rtag and PC new tag (as written to CSRs G8 and G9). In such acase, multiplexers M1-M7 may be used to select the foregoing new valuesfrom Op1data 1051 as inputs, respectively, for CSRs G1-G7.

Generally, Op1data 1051 denotes output from the metadata processor tothe PUMP and R data 1053 denotes output from the PUMP to the metadataprocessor. Also, element 1052 denotes inputs to the PUMP from theprocessor executing user code (e.g., as part of normal instructionprocessing) where values for element 1054 are equal to those as in 1052when performing normal PUMP verification (e.g., verify whether currentinstruction is allowed using policy rules).

Referring to FIG. 33, shown is an example 1060 illustrating PUMPprocessing stages in combination with a 6 stage processor pipeline inone embodiment in accordance with techniques herein with a RISC-Varchitecture with branch prediction. The example 1060 illustrates a 6stage pipeline with stage 1 including fetching the next instruction tobe executed (e.g., storing fetched instruction in I cache 1063 a) andbranch prediction, stage 2 denoting the decode instruction stage, stage3 including obtaining values from registers (e.g., register read) andbranch resolution for the current instruction, stage 4 includinginstruction execution (e.g., execute fast ALU operations and launchmulti-stage operations such as floating point (FP), integermultiplication and division), stage 5 including receiving responses tomulti-stage operations and requesting memory operands, and stage 6including committing instructions (e.g., storing result to destinationand in data cache 1063 b as denoted by 1069), and handling exceptions,traps and interrupts. Also shown in the example 1060 are the PUMPprocessing stages. Element 1062 indicates that the opgrp/care tablelookup may be performed in stage 3 with the output 1062 a provided as aninput in stage 4 to the PUMP hash 1064. Other inputs to PUMP hash 1064include the Mtag 1061 (e.g., tag of memory location that is an operandfor the current instruction) and other tag values 1062 b whereby inputs1061 and 1062 a-b are used to determine output 1064 a denoting a cacheaddress or location in the PUMP rule cache 1066. Examples of other tagvalues 1062 b of instruction operands, the PC, current instruction, andthe like, are described elsewhere herein and may be used in connectionwith determining a location in the rule cache 1066 for the currentinstruction (e.g., FIG. 22). Element 1068 denotes cache rule missdetection based on outputs 1066 a of the PUMP processing from stage 5.Outputs 1066 a may include an indicator as to whether there was a rulecache miss for the current instruction. If 1066 a reports a potentialhit, 1068 determines if the hit is a true hit or a false hit, turningfalse hits into misses. Element 1066 b denotes the PUMP outputs to stage6 in the case where there is no rule cache miss and there is a rule incache matching the current instructions. Outputs 1066 b may include PCnew tag and R tag. It should be noted that the PUMP stages of theexample 1060 may be varied. For example, the opgroup/care lookup 1062may be performed in stage 4 rather than stage 3 with determination of aPUMP rule cache location and lookup both done in stage 5 (e.g.,depending on the particular PUMP rule cache implementation).

In connection with non-memory operations, the Mtag is not needed as aninput to the PUMP stage and the PUMP may continue performing processingwithout it. In the case of a memory operation instruction, the PUMPstalls until the Mtag has been retrieved from memory. Alternatively, anembodiment may perform Mtag prediction as described elsewhere herein.Consistent with discussion elsewhere herein, the PC new tag needs toprovided back to the stage 1 such as illustrated and described inconnection with FIG. 1. As long as the instruction commits, the PC newtag is the appropriate PC tag for the next instruction. If the currentinstruction does not commit (e.g., no rule cache hit), the PC new tag(as passed back to stage 1) is determined by the rule cache misshandler. When a trap handler starts or a context switch is performed(e.g., PC restore), the tag comes from saved PC.

As described herein, an embodiment may associate a single tag with eachword. In at least one embodiment, the word size associated with each tagmay be 64 bits. The contents of the tagged word may contain, forexample, an instruction or data. In such an embodiment, the size of asingle instruction. However, an embodiment may also support instructionswhich are a different size other than 64 bits. For example, anembodiment may be based on the RISC-V architecture which, as describedin more detail elsewhere herein, is an open source instruction setarchitecture (ISA) based on established reduced instruction setcomputing (RISC) principles. An embodiment using the RISC-V architecturemay include instructions of multiple different sizes such as, forexample, 32 bit instructions as well as 64 bit instructions. In such acase, an embodiment in accordance with techniques herein may associate asingle tag with a single 64 bit word where the single word may thereforeinclude one 64 bit instruction or two 32 bit instructions.

Referring to FIG. 34, shown is an example 200 of tags that may beassociated with instructions in an embodiment in accordance withtechniques herein. Element 201 illustrates the case noted above where asingle tag 202 a is associated with a single instruction 204 a. In atleast one embodiment, the size of each of 202 a and 204 a may be a 64bit word. Element 203 illustrates an alternative also noted above wherea single tag 202 b is associated with two instructions 204 b and 204 c.In at least one embodiment, the size of 202 b may be a 64 bit word, andthe instructions 204 b and 204 c may each be 32 bit instructionsincluded in the same 64-bit instruction word 205 associated with tag 202b. More generally, it should be noted that there may be more than 2instructions in a single tagged instruction word depending on theinstruction size(s) used in an embodiment. If, as illustrated by element203, the granularity of tagging does not match the granularity ofinstructions, then multiple instructions are associated with a singletag. In some instances, the same tag 202 b may be used for each of theinstructions 204 b 204 c. However, in some instances, the same tag 202 bmay not be used for each of the instructions 204 b, 204 c. In followingparagraphs, each of the multiple instructions, such as 204 b and 204 c,included in a single instruction word associated with a single taggedword may also be referred to as a subinstruction.

Thus what will now be described are techniques that may be used in anembodiment in connection with multiple subinstructions in the sameinstruction word whereby a different tag may be used in connection witheach of the multiple subinstructions.

Referring to FIG. 35, shown is an example illustrating instructions andtags that may be used in an embodiment in accordance with techniquesherein. The example 220 includes a single 64-bit instruction word 205that includes two 32 bit subinstructions 204 b and 204 c. Tag 202 b maybe the tag on the instruction word 205 as described above in the example200. In at least one embodiment in accordance with techniques herein,the tag 202 b of the instruction word 205 may be a pointer 221 toanother memory location 222 that includes a pair of tags where the pairincludes a tag for each of the subinstructions 204 b-c of theinstruction word 205. In this example 220, the pair of tags 222 includes222 a denoting a first tag, tag1, for substruction1 204 b, and alsoincludes 222 b denoting a second tag, tag2, for substruction2 204 c. Inat least one embodiment, each tag 222 a-222 b of the pair 222 may be anon-pointer tag (e.g., scalar), may be a pointer tag to yet anothermemory location including information used by the PUMP for processing asdescribed herein, or may otherwise be a more complex structure includingone or more non-pointer fields and/or one or more non-pointer fields.For example, tag 1 222 a may be a pointer tag for subinstruction 1 204 aand tag2 222 b may be a pointer tag for subinstruction 2 204 b. Asillustrated in 220, element 223 a denotes tag1 222 a pointing to oridentifying another memory location 224 a including information used bythe PUMP for processing subinstruction1 204 b, and element 223 b denotestag2 222 b pointing to or identifying another memory location 224 bincluding information used by the PUMP for processing subinstruction2204 c. It should be noted that, depending on the embodiment and thesubinstructions, each of 224 a and 224 b may be a non-pointer, may beyet another pointer to a memory location, or may be complex structureincluding some combination of one or more pointers and one or morenon-pointers.

In an embodiment having multiple subinstructions within the sameinstruction word 205, an additional input may be provided to the PUMPindicating which of the subinstructions included in the instruction word205 is being executed at a point in time. For example, where there are 2subinstructions 204 b-c in the instruction word 205, the additionalinput to the PUMP may be 0 or 1 indicating, respectively, whethersubinstruction 1 204 b or subinstruction2 204 c is being executed at aparticular point in time. In at least one embodiment consistent withdiscussion elsewhere herein based on the RISC-V architecture, a CSR(such as the ssubinstr CSR described elsewhere herein) may be definedwhich records or stores the additional input (denoting whichsubinstruction is being executed) to the PUMP. In at least oneembodiment, the PUMP may normally receive the foregoing additional inputfrom the data path (e.g., from the code execution domain) without use ofa CSR. However, on a rule miss, the foregoing additional input may berecorded in a CSR so that the metadata processing domain in which therule miss handler is executing may obtain the foregoing additional input(e.g., the CSR value for the foregoing additional input is provided tothe PUMP on a rule insertion).

To further illustrate, an embodiment may include subinstructions whichprovide for transfer of control between two locations in a program.Examples of such subinstructions may be those that provide for jumping,branching, returning or more generally transferring control from asource location in the code to a target (e.g., sink or destination)location in the code. In connection with CFI or control flow integritydescribed elsewhere herein, it may be desirable to have the PUMPimplement rules of a CFI policy to limit or control transfers betweenlocations to only those supported by the program. For example, considera case where a transfer of control is made from a source location incode having tag T1 to a target location in code having tag T2.Information used by the PUMP in enforcing the CFI policy may be a listof valid source locations which are allowed to transfer control to T2.In an embodiment of the CFI policy, two rules may be used to provide twochecks of two instructions or opcodes when transferring control from thesource to the target location. Consider a pseudo-code representation ofthe transfer or call as illustrated in the example 230 of FIG. 36. Inthe example 230, a call may be made transferring control 231 a from asource location in foo routine 231 to a target location in routine bar233. Specifically, control may be transferred 231 a from the sourcelocation X1 232 having tag T1 to the target location X2 234 having tagT2. The target location X2 may be a first instruction in the body ofcode 233 a of routine bar. The rules of the CFI policy may be used tocheck with the transfer from 232 to 234 is allowed or valid. In at leastone embodiment, 2 rules of the CFI policy may be used each performing acheck to ensure the transfer of control from 232 to 234 is valid. Theinstruction at the source location X1 is a branch point or source pointfrom which control is transferred to the target. At the source (e.g.,prior to executing the instruction at the source location X1 232), afirst rule may be used to mark or set the tag of the PC to denote thesource location. For example, the first rule may mark or set the tag ofthe PC to be the address X1 to denote the source location. Subsequently,prior to executing the instruction at the target location X2 234), asecond rule may be used to check whether the source location X1 is avalid source location from which control is allowed to be transferred tothe target location X2.

In at least one embodiment, the check of the second rule may beperformed by determining whether the marked tag of the PC (as set by thefirst rule) identifying the source location 232 (e.g., which denotes thesource location address X1) identifies a valid source location fromwhich control may be transferred to target location 234. In such anembodiment, the second rule may be supplied with a defined list denotingall valid source locations which are allowed to transfer control to thetarget location 234. In at least one embodiment, the defined list mayidentify valid source locations, for example, by their addresses such asX1 noted above.

Referring to FIG. 37, shown is an example 240 illustrating tags that maybe used in connection with subinstructions of source and targetlocations in an embodiment in accordance with techniques herein. Theexample 240 includes element 203 denoting the single tag 202 b specifiedfor 2 subinstructions 204 b-c of a single instruction word as describedabove. The tag 202 b on the instruction word may point to the tag pair242 denoting the two tags 242 a-b, respectively, for the twosubinstructions 204 b-c. Each of two tags 242 a-b may generally be apointer to information used by PUMP rules for CFI validation inconnection with the source or target location depending on theparticular subinstruction associated with each of the two tags 242 a-b.

The example 240 illustrates structures in one embodiment where the twosubinstructions 204 b-c are target locations. The subinstruction tag 242a points 243 a to a location of a structure 245 including a source idfield 245 a and an allowed source set field 245 b. The source id field245 a may be null in the case where the subinstruction 204 b is not asource location, such as the case here with subinstructions 204 b is atarget location. The source set field 245 b may be a pointer to alocation including a list structure 247 identifying one or more validsource locations which are allowed to transfer control to the particulartarget location including subinstruction 204 b. In at least oneembodiment, the list structure 247 may include a first element denotinga size or number of valid source locations. Thus size 247 a of “n” (nbeing an integer greater than 0) denotes the number of source locationsdenoted by elements 247 b-n in the list 247. Each of elements 247 b-nmay identify a different valid source location which can transfercontrol to the target location including subinstruction 204 b. In atleast one embodiment, each of the allowed sources 247 b-n may be ascalar or non-pointer that is, for example, the address of one of thevalid source locations.

In the example 240, elements 243 b, 246 and 248 used with subinstruction2 204 c are respectively similar to elements 243 a, 245 and 247 as usedwith subinstruction 1 204 b. Generally, in such an embodiment using thestructures of 240, any item that does not exist may be assigned a nullor zero value. If the instruction word 205 includes a pair ofsubinstructions 204 b-c that are neither source nor destinationlocations, the tag 202 b may be null (e.g., or otherwise identify anon-pointer or other pointer that does not point to a structure 242). Ifone of the subinstructions 204 b-c is neither a source nor a targetlocation of a transfer, its associated tag in 242 is null. For example,if subinstruction 204 b is neither a source nor a target location butsubinstruction 204 is a target, then 242 a may be null and 242 b may beas illustrated in the example 240. If a subinstruction 204 b-c is not asource location, its source id is null (e.g., since 204 b-204 c in theexample 240 are target locations, both 245 a and 236 a are null). If asubinstruction 204 b-c is not a target location, its allowed source setfield pointer is null. For example, if subinstruction 204 b identified asource location rather than a target location, source id 245 a wouldidentify the address of the source location instruction and 245 b wouldbe null.

To further illustrate, reference is made to FIG. 38 to another example250 using the structures such as described in the example 240 with thedifference that 251 a, a first of the subinstructions, is neither asource nor a target location, and 251 b, a second of thesubinstructions, is a target location where control may be transferredfrom any of 3 valid source locations. Elements 251 a-b may denote two 32bit subinstructions included in a single tagged word having tag 251. Thetag 251 may be pointer 1228 identifying a location in memory includingthe structure 252 with a pair of tags for the subinstructions 251 a-b.Element 252 may be similar to 242 of the example 240. Element 252 a maybe a tag pointer for the subinstruction 251 a and element 252 b may be atag pointer for sub instruction 251 b. Since subinstruction 251 a isneither a source nor a target location, 252 a is null as denoted by thezero. Since subinstruction 251 b is a target location, 252 b is apointer 1238 to structure 254. Element 254 may be similar to 246 of theexample 240. Element 254 a is a source id field (like 246 a) and element254 b is an allowed source set field (like 246 b) including a pointer(address 1248) to an allowed source set structure 256 (similar to 248 ofexample 240). Since subinstruction 251 b is only a target location andnot a source, 254 a source id is null. Element 256 may be similar to 248of the example 240. Element 256 a may be a size field (like 248 a)denoting a number of valid source locations. Element 256 b-d may denotethe valid source ids which may be, for example, addresses of validsource location instructions. In this example, 256 a indicates thatthere are 3 valid source locations having addresses 50 bc, 5078, 5100stored, respectively, in entries 256 b-d. In connection with theforegoing, it should be noted that generally an instruction may be botha target and a source so that being a target does not mean that sourceid will always be null. If, for example, an instruction is both a targetand a source, source id will not be null and the instruction's tag wouldinclude the list of allowable/allowed sources.

It should be noted that the addresses of the source locations such as inentries 256 b-d, and more generally, in any allowed source of theallowed source set (e.g., any of 248 b-n of 248 of the example 240) maybe a byte level address granularity.

In a manner similar to that just described for multiple instructions(also referred to as subinstructions) included in a single tagged word,an embodiment may allow access to data portions which are less than asingle tagged word of data. For example, an embodiment may includeinstructions which access data at the byte level and it may be desirableto provide byte level tagging so that each byte may have its ownassociated tag in manner similar to providing a different tag for eachof the multiple subinstructions included in a single tagged word. Infollowing examples, reference is made to providing byte level taggingwhere each of 8 bytes included in a 64 bit word may have its ownassociated tag. However, more generally, techniques herein may be usedto provide for sub-word tagging for any number of multiple data itemsincluded in a single tagged word. In such cases, the tag associated withthe tagged data word may be a pointer to a structure identifying thebyte level tags for the bytes of the tagged data word.

Referring to FIG. 39, shown is an example 260 of byte level tagging thatmay be used in an embodiment in accordance with techniques herein.Element 262 denotes a tag 262 a associated with a tagged 64 bit word 265where the word 265 includes 8 bytes denoted as B1-B8. Tag 262 a may be apointer pointing 261 to a memory location of structure 266 includingtags for each of the bytes B1-B8 of the data word 265. The structure 266may include a first field 265 a that is a size field indicating a numberof remaining entries in the structure. Each subsequent entry in thestructure may include a tag value and denote the one or more bytes ofthe word 265 having that particular tag value. In this example, size 265a is 8 where each of the bytes B1-B8 of 265 have a different tag value.Element 266 a-h respectively denote tag values for bytes B1-B8 of theword 265.

Referring to FIG. 40, shown is a second example 267 of byte leveltagging that may be used in an embodiment in accordance with techniquesherein. Element 262 denotes a tag 262 a associated with a tagged 64 bitword 265 where the word 265 includes 8 bytes denoted as B1-B8. Tag 262 amay be a pointer pointing 268 a to a memory location of structure 268 bincluding tags for each of the bytes B1-B8 of the data word 265. Thestructure 268 b may include a first field 265 b that is a size fieldindicating a number of remaining entries in the structure. Thus, 265 bis similar to 265 a of FIG. 39. Each subsequent entry in the structure268 b may include a tag value and denote the one or more bytes of theword 265 having that particular tag value. In this example, size 265 bis 7 denoting the 7 subsequent entries 266 a-226 f and 268 c. Element266 a-f are as described in connection with the example 260 of FIG. 39.Element 268 c indicates that tag 7 is the tag for both bytes B7 and B8.Thus, the structure 268 b includes one less entry than the structure 266of FIG. 39 since, in the example 267, both bytes B7 and B8 have the sametag value of tag 7. In this manner, the structure (e.g., 268 b) pointedto by a tag (e.g., 262 a) of a data word may have a varying number ofentries as needed depending on the particular byte level tags.

It should be noted that the particular level of data access granularitymay vary with the particular architecture and instruction set in anembodiment. The foregoing may be used to provide byte level tagging inan embodiment which allows byte level data access. As a variation, anembodiment may support data access at a different level of granularityand techniques herein may be readily extended to any subword tagginglevel of granularity.

Similarly, the examples 260 and 267 illustrate one example of a datastructure that may be used to hold the byte level or other sub word datatagging. As a variation, an embodiment may use a tree or otherhierarchical structure to specify byte level tags for bytes of a singletagged data word. The tree or other hierarchical structure representingthe byte level tags may be similar the hierarchical structure describedherein with for storing word-level tags, for example, in connection withelements 100, 120, 130 and 140, respectively of FIGS. 78-81 describedelsewhere herein.

To further illustrate, an embodiment may use a tree structure torepresent byte level tags as in the example 270 of FIG. 41. In theexample 270, element 262 may denote a tag 262 a associated with taggedword 265 including bytes B1-B8. Tag 262 a may be a pointer or address toa tree structure representing byte level tags for B1-B8 265. Forexample, tag 262 a may point to a location of root node 272 of the treestructure. The tree structure in this example may include root node 272at level 1, nodes 274 a-b at level 2, nodes 276 a-d at level 3 and nodes278 a-h at level 4. Each node of the tree may be associated with a byterange of one or more bytes. The leaves of the tree may denote the bytelevel tags for the bytes B1-B8. A non-leaf node of the tree thereforedoes not specify a tag value but rather indicates that one or moredescendant nodes at one or more lower levels need to be consulted todetermine the byte level tags for the byte range associated with thenon-leaf node. A leaf node may denote a homogenous or same tag value fora range of multiple bytes of 265. Each non-leaf node may include a leftpointer to the non-leaf node's left child node and a right pointer tothe non-leaf nodes' right child node. Each of the child nodes of aparent node may represent a partitioning of the byte range associatedwith the parent node.

The example 270 illustrates a tree structure where there are nohomogeneous byte level tags and each of the bytes B1-B8 of 265 has adifferent tag value. In a manner consistent with discussion elsewhereherein (e.g., with elements 100, 120, 130 and 140, respectively of FIGS.78-81), an embodiment may omit descendant nodes from a subtree if thesubtree has as its root a first node denoting a homogeneous tag valuefor the byte range associated with the first node. For example, tofurther illustrate, reference is made to FIG. 42. In the example 280,element 262 may denote a tag 262 a associated with tagged word 265including bytes B1-B8 as descried above. Tag 262 a may be a pointer oraddress to a tree structure representing byte level tags for B1-B8 265.In this example 280, each of the bytes B1-B8 has the same tag T1 andtherefore the tree structure need only include the root node 281. Asbyte level tags for bytes B1-B8 may be modified or changed over time,the tree structure or other structure pointed to by tag 262 a may beaccordingly updated to reflect such byte level tag modifications.

In an embodiment providing byte level tagging, or more generally subwordtagging, within the same data word 265, an additional input may beprovided to the PUMP indicating which one or more byte level tags(corresponding to which one or more of the bytes included in the word265) are being referenced. For example, with byte level tagging wherethere are 8 bytes B1-B8 in a single tagged data word 265, the additionalinput to the PUMP may be a bitmask of 8 bits where each of the 8 bits isassociated with a different one of the bytes B1-B8 and denotes whetherto use the byte level tag for the particular byte of the word 265. As avariation, an embodiment may denote the one or more bytes by specifyinga byte range, such as starting byte and length or size (e.g., bytesB4-B8 by specifying starting byte B4 and denoting a size or length of5). In at least one embodiment consistent with discussion elsewhereherein based on the RISC-V architecture, a CSR may be defined whichrecords or stores the additional input denoting which one or more bytelevel tags for the one or more bytes B1-B8 are to be used by the PUMP.The additional input may be, for example, the bitmask or other suitablerepresentation identifying the particular byte level tags used by thePUMP. In at least one embodiment, the PUMP may normally receive theforegoing additional input denoting which one or more bytes are to beused as an input from the data path (e.g., from the code executiondomain) without use of a CSR. However, on a rule miss, the foregoingadditional input may be recorded in a CSR so that the metadataprocessing domain in which the rule miss handler is executing may obtainthe foregoing additional input (e.g., the CSR value for the foregoingadditional input is provided to the PUMP on a rule insertion).

As discussed elsewhere herein, at the policy level many instructions maybe treated in a similar manner. For example, add and subtractinstruction operation codes or opcodes may typically treat theirmetadata the same whereby both opcodes may behave similarly at the rulelevel for a particular policy by considering the same tag inputs to thePUMP and the same tag outputs propagated by the PUMP. In such a case,add and subtraction opcodes may be grouped together in a singleoperation group or “opgroup” so that the same set of rules may be usedfor all opcodes in that particular opgroup. How opcodes are groupedtogether is policy dependent and thus may vary with policy. In oneembodiment, a translation or mapping table may be used which maps aparticular opcode to its associated opgroup on a per policy level. Inother words, a different mapping table may be created for each policy(or specified group of multiple policies having the same opcode toopgroup mappings) since the mappings may vary per policy.

For a particular opcode, the translation or mapping table may determinethe opgroup as noted above and may also determine additional informationfor the particular opcode. Such additional information may include thecare/don't care bit vectors as also discussed elsewhere herein which mayindicate which PUMP inputs and PUMP outputs (e.g., input tags andpropagated output tags) are, respectively, actually used as inputs forrule processing and propagated as a relevant output of rule processingfor a particular opcode. The don't care bit vectors may be determinedwith respect to any PUMP input and output in an embodiment. In oneembodiment, the don't care bit vector may indicate which input tags andoutput tags are relevant and may also indicate which particular opcodebits are actually used for a particular opcode. This is described belowin more detail with respect to the RISC-V architecture and instructionformats but may also be more generally used in connection with othersuitable instruction formats of different architectures. The forgoingtranslation or mapping table including opgroups and care/don't care bitsfor particular opcodes (e.g., element 422 of example 420 discussedbelow) may also be referred to as the opgroup/care table elsewhereherein.

RISC-V has multiple different instruction formats each using a differentset of instructions bits for the opcode. Referring to the example 400 ofFIG. 43, shown are bits of an instruction that may be included indifferent bit encodings for different opcodes in an embodiment usinginstructions of the RISC-V architecture. Generally, the RISC-Varchitecture includes multiple instruction formats where different bitsof the instruction may be used as part of the opcode encoding. With a 32bit instruction, a total of up to 22 bits may be used to represent anencoding of an opcode. Element 404 represents portions of an instructionin the RISC-V architecture that may be used to represent a bit encodingfor a particular opcode depending on the instruction format. Element 404includes 3 fields of bits —404 a-404 c—that may be used in encoding aparticular opcode. Element 404 a indicates a first opcode field, opcodeA, of 7 bits. Element 404 b indicates a second opcode field, funct3, of3 bits. Element 404 a indicates a third opcode field, funct12, of 12bits. Depending on the instruction (e.g., such as with system calls),the opcode encoding may include up to all 22 of the bits denoted by 404a-c. More specifically, in RISC-V, an opcode may be encoded using justthe 7 bits of 404 c, using 10 bits of only 404 b and 404 c (exclude 404a), or using all 22 bits of 404 a-c. As a further variation, aninstruction of the RISC-V architecture may have an opcode encoding usingfields as denoted by 402. Element 402 includes the two fields of bits404 b and 404 c as discussed above. Additionally, rather than use all 12bits of funct12 404 a in the opcode encoding, an instruction may useonly 7 of the 12 bits as denoted by funct7 402 a. Thus, as yet anotherpossibility, an opcode may have an encoding using fields 402, 404 b and404 c as illustrated by element 402.

Illustrated in FIG. 44 is an example 420 illustrating a mapping ortranslation table that may be used in an embodiment in accordance withtechniques herein. As discussed above, an opcode 421 may be provided asan input or index into the opcode mapping table 422 to lookup ordetermine mapped outputs 424 for the opcode 421. The mapped outputs 424may include the opgroup and care/don't care bit vectors for PUMP inputsand outputs for the particular opcode 421. In an embodiment based on theRISC architecture and instruction formats, the opcode may potentiallyhave up to a 22 bit encoding. However, using such a large 22 bit opcodeas an index into the table is unreasonable due to the large number ofentries needed to accommodate the 22 bit opcode (e.g. table may includean entry for each opcode indicating its associated opgroup andcare/don't care bit vector information resulting in millions of entriesfor the 22 bit opcode). To reduce the size of the table 422 in such anembodiment, the table 422 may be indexed using only a portion of the 22bit opcode fields. For example, in at least one embodiment, the opcode421 input may be 10 bits of the opcode as denoted by elements 404 b and404 c in the example 400. Thus, table 422 may be indexed using opcodebits of 404 b and 404 c of an opcode to determine the opcode's opgroupand associated care/don't care bit vectors.

In such an embodiment, the remaining 12 opcode bits of funct 12 404 a ofan instruction may be provided as an input to the PUMP where appropriateportions of 404 a are masked for the particular opcode. Informationregarding which particular bits of funct12 404a should/should not bemasked for a particular opcode may be included in the care/don't carebit vector information output from the mapping table 422 lookup for theopcode. In at least one embodiment based on the RISC-V architecture, thecare/don't care bit vector information may indicate one of the followingwith respect to the 12 opcode bits of funct 12 404 a for an opcode:

-   -   1. all 12 bits may be masked since no bits of 404 a are used;    -   2. 7 of the 12 bits, as denoted by 402 a, are used where the        bottom most 5 bits of 404 a (e.g., bits 20-25) are masked out;        or    -   3. all 12 bits of 404 a are used and therefore there is no        masking of an bits of 404 a.

Also, in such an embodiment, the 12 opcode bits of funct12 404a may berecorded or stored in a CSR, such as sfunct12 CSR described elsewhereherein, provided as a PUMP input in connection with performing ruleinsertion into the PUMP. In at least one embodiment, the PUMP maynormally receive the foregoing opcode bits from the data path (e.g.,from the code execution domain) without use of a CSR. However, on a rulemiss, the foregoing may be recorded in a CSR so that the metadataprocessing domain in which the rule miss handler is executing may obtainthe foregoing as an input (e.g., the CSR value is provided as an inputto the PUMP on a rule insertion).

In at least one embodiment in accordance with techniques herein,multiple user processes may execute using a virtual memory environmentwhere physical pages are mapped into a user process address space.Techniques herein may be utilized to allow sharing of physical pages ofmemory among multiple user processes where the same set of one morephysical pages that may be simultaneously mapped into multiple userprocess address spaces. In at least one embodiment, tags used by suchprocesses for which the sharing is allowable may be characterized asglobal having the same value and meaning or interpretation across userprocess address spaces.

Referring to FIG. 45, shown is an example 430 illustrating sharing ofphysical pages between processes in an embodiment in accordance withtechniques herein. The example 430 includes process P1 having addressspace 434 and process P2 having address space 436. Element 434 maydenote the virtual memory process address space or range 0 through MAX,where MAX denotes the maximum virtual memory address used by P1 and 0denotes the minimum virtual address used by P1. As known in the art,physical pages of memory 432 may be mapped into a virtual address spacesuch as 434 where the contents of the mapped physical page may beaccessed by P1 using the mapped virtual addresses of such mappedphysical pages. For example, physical page A 432 a may be mapped into asubrange X1 of P1's virtual address space. Process P1 may, for example,read a data item or instruction from a location in page A 432 a byreferencing a particular virtual address in the subrange X1.

Similarly, physical pages of memory 432 may be mapped into virtualaddress space 436 where the contents of the mapped physical page may beaccessed by P2 using the mapped virtual addresses of such mappedphysical pages. For example, physical page A 432 a may be mapped into asubrange X2 of P2's virtual address space. Process P2 may, for example,read a data item or instruction from a location in page A 432 a byreferencing a particular virtual address in the subrange X2.

The tags 431 may denote the tags on the memory locations of page A 432where such tags may be used by the PUMP in connection with ruleprocessing as described herein. Since page A 432 is shared by both P1and P2 via the mapping as illustrated, the same set of tags 431 are alsoused by the PUMP in connection with executing instructions of both P1and P2. In such an embodiment, the tags 431 may be characterized asglobal tags shared by both P1 and P2. Additionally, in at least oneembodiment, the global tags 431 shared by multiple processes P1 and P2are interpreted in a similar manner such as using the same rules andpolicies. For example, a first tag having a value of 100 may beassociated with a first memory location in 432 a. The first tag maydenote a value representing a coloring of the first memory location usedin connection with rules of policy which determine whether it isallowable for a particular executing instruction to perform an operationreferencing the first memory location, or its contents. The first tagmay be interpreted as the same color by the rules in connection withinstruction execution of both P1 and P2. For example, the tag value of100 needs to be interpreted as the same color by the rules in connectionwith both P1 and P2. Furthermore, the same set or instance of policiesand rules may be used by the PUMP for both P1 and P2.

In such an embodiment which uses global tags on shared memory asdescribed above, it may be desirable to also allow for furtherdifferentiating or allowing different access, authority or operations ona per process basis. For example, assume that page A 432 a includes datashared by both P1 and P2. However, it may be desirable to allowdifferent operations or access with respect to the shared data of 432 aon a per process basis even though global tags are used to tag theshared page A 432 a. For example, process P1 may have write access topage 432 a and process P2 may have read-only access to page 432 a.However 432 a may be a shared memory page tagged with global tags. Insuch an embodiment with global tags on the shared page, the same policyand set of rules may be used in connection P1 and P2 where differentread and write access capabilities for each process may bedifferentiated using different tag values on the PC. For example,process P1 may be include a first instruction which performs a write toa memory location in 432 a and the current PC tag has a value of X.Rules of an access policy may perform the following logic:

-   -   if PCtag=X, then allow write    -   if PCtag=Y then allow read-only        In such a case, the PC tag has a value of X which is interpreted        by the rules to allow write access for process P1 and thus P1 is        allowed to execute the first instruction. Process P2 may be        executing a second instruction which performs also performs a        write to a memory location in 432 a and the current PC tag has a        value of Y. In such a case, the PC tag has a value of Y which is        interpreted by the rules to not allow write access and rather        allow read-only access for process P2 and thus P2 is not allowed        to execute the second instruction.

Thus, in at least one embodiment, the PC tag may be used to encodeprivilege, access or authority that may differ per process whereby theparticular allowed privilege, access or authority may be represented bydifferent PC tag values.

An embodiment may specify a particular PC tag value to be used for eachprocess in any suitable manner. For example, privileged code may executeas part of operating system startup or initialization which initiallyspecifies a PC tag value to be used for a particular process. As avariation, an embodiment may perform a mapping operation as part ofmapping the shared page A 432 a into a process address space. The rulesapplied by the operating system when performing the mapping maypropagate or produce a particular PC tag as an output that denotes adesired access, privilege or authority based on the particular process.

In this manner, the same set of rules may be used with shared pageshaving global tags where the rules encode the logic for the differencein access, authority or privilege based on the PC tag. It should benoted that the PC tag may also be a pointer to a memory location wherebythe pointer tag points to a structure including different tag values fordifferent policies in a manner as described herein in connection withother tags. In this manner, the same set of PC tag values may be used todenote different capabilities for a process that may vary with policy.For example, the PC tag value of X as described above with P1 may have afirst use as described above with a memory safety policy or data accesspolicy for shared regions. The same PC tag value of X may have a seconduse and meaning imparted by rules of a second different policy, such ascontrol flow integrity (CFI).

Aspects of the CFI policy are described herein that may be used inconnection with restricting control transfers based on a staticdefinition of allowable calls, jumps, return points, and the like.However, an additional aspect or dimension that may be included in a CFIpolicy relates to enforcement of dynamic or runtime call informationthereby further refining the conditions under which a control transferthat is a return may be made. To further illustrate, reference is madeto the example 500 of FIG. 46 which includes routines foo 502, bar 504and baz 506. Routine Foo 502 may include a call instruction at addressX1 that calls routine bar resulting in a runtime transfer 501 a ofcontrol to bar 504. Routine bar 504 then includes a return instructionwhich returns 501 b control to routine bar to address X2. Thus, X2denotes the return point address or location of the instruction inroutine foo following the call to routine bar at X1. Routine Foo 502 mayinclude a second call instruction at address Y1 that calls routine baz506 resulting in a runtime transfer 501 c of control to baz 506. Routinebaz 506 then includes a return instruction which returns 501 d controlto routine bar to address Y2. Thus, Y2 denotes the return point addressor location of the instruction in routine foo following the call toroutine baz at Y1.

A static CFI policy may, for example, allow all potential control flowsbetween any two transfer points without further restricting controlflows or transfers based on the current runtime stack or call chainreflecting the dynamic runtime control flow aspects. For example, Forexample, if foo 502 can call bar 504 as illustrated in 500, there is astatically allowed control flow from bar back to address X2 of theinstruction after the call of bar at X1 in foo. However, if foo has notbeen invoked, or has only, so far, invoked another call to somethingthat should return before the bar call, it should not be possible toexercise the return link to return to X2. As another example with theruntime execution as illustrated in the example 500, it should not bepossible for a call to Bar 504 through 501 a to return to Foo 502 at Y2through 501 d.

What will now be described are techniques that may be used in connectionwith extending rules of a CFI policy to enforce a dynamic CFI returnpolicy controlling return flow path control. For the dynamic CFI returnpolicy to ensure that a return to a particular return location such asX2 is valid only when made subsequent to a particular call orinvocation, such as the call to bar at X1, the dynamic CFI return policymay store information, such as in one or more tags, when the call ismade in order to rule out an invalid return. As known in the art, when acall is made, such as using a JAL (jump and link) instruction of theRISC-V instruction set, a return address is saved in a return addressregister, RA. The RISC-V instruction set also includes a JALR (jump andlink register) instruction which is an example of a return instruction.In one aspect, the saved return address in RA register from a JAL may becharacterized as a “capability” to return to that point. In at least oneembodiment, the JAL instruction may be tagged with a tag that causes arule to push a suitable tag capability onto the resulting returnaddress. For example, with RA as the return address register, a rule mayplace a tag on the RA register which indicates that the RA registerincludes a valid or suitable return address and, at a later point, theaddress of the RA register may be used as a return point to whichcontrol may be transferred. In other words, the tag on the RA registergives permission for the address in RA to be used as a return addresswhich is loaded into the PC to execute the return transfer of control.When loading the PC with the address of RA, the RA tag may also bestored as the PC tag by rules of the CFI policy.

To further illustrate techniques that may be used to limit control flowon returns, an embodiment may code tag each return point (e.g., X2, Y2)with a dynamic-CFI-tag, such as expect-A. Also, code tag each JALinstruction (or call instruction) causing a rule evaluated for the JALinstruction to tag the return address in the RA register (where thereturn address is calculated by the JAL) with the appropriatedynamic-CFI-return-to-A tag. For each return, such as each JALRinstruction that uses the RA register tagged with thedynamic-CFI-return-to-A tag, a PUMP rule propagates the tag(dynamic-CFI-return-to-A tag) onto the PC as may be performed inconnection with other static CFI policy rules. The rules of the CFIpolicy may embody logic that checks the RA register used for the returninstruction. If the RA register used for the return is not tagged withdynamic-CFI-return-to-A tag, then it is known that the RA register doesnot include a valid return address allowed for use with the JALRinstruction. At the return point (e.g., X2 and Y2), rules may embodylogic whereby when the expect-A code tag is encountered (e.g., as thetag on the instruction at X2), check that the PC is tagged withdynamic-CFI-return-to-A, and clear the CFI-return-to-A tag from the PC.

As a consequence of the above, code is prevented from returning to justany return address. Furthermore, if the return address is copied toanother location, such as another register, the rules can prevent thecopied value from retaining the return authorization capability; thisprevents code form making copies of the return address in registers thatcan be used to perform multiple returns for the same call. As anotherconsequence of the above, if a valid return address (properly tagged) onthe stack is overwritten with a new address (not properly taged) andthen an attempt is made to return to the new address, the return isprevented.

An embodiment may also include rules in order to prevent or furtherlimit the ability to use the dynamic-CFI-return-to-A tag more than once.As a first implementation, an embodiment may use rules that restrictwhere the return address (as stored in the RA register tagged with thedynamic-CFI-return-to-A tag) may be written or copied. For example, anembodiment may use rules that only allow the return address of theappropriately tagged RA register to write the return address to thestack in properly code-tagged function code. As a second alternativeimplementation, an embodiment may include rules that use PC state (e.g.,PC tag) and atomic memory operations to make the return address linear(e.g., follow or occur subsequent to a call). For example, performing acall sets the PC tag to denote a valid-return-address. Rules may onlyallow a return if the PC tag is set to valid-return-address. Additionalrules may be used that, when writing a return address to memory, setsthe PC tag to no-return-address. Rules may be used that, when copyingthe return address to a target register, may set the PC tag tono-return-address, and the target register is not tagged as avalid-return address. Rules may be used that, when an arithmeticoperation is performed using a return address from an RA register, theresult is not tagged as a valid return address. Rules may be used thatonly allow recovering a return address from memory with an atomic swapoperation with a non-return-address (e.g., where the PC tag is set tovalid-return-address).

An embodiment may further define rules to provide a stack protectionpolicy. In one aspect, the stack protection policy may, in part, beviewed as an extension of one or more other policies, such as memorysafety where the rules may use tags of both instructions and data forpolicy enforcement. It should be note that in following discussion andelsewhere herein, terms such as routine and procedure may be usedinterchangeably and more generally refer to a callable unit of codethat, when invoked, results in creation of a new stack frame on the callstack. Other names that may also be used for a callable unit of code mayinclude function, subroutine, subprogram, method, and the like.

Referring to FIG. 47, shown is an example 520 illustrating a call stackof frames for runtime invocations in an embodiment in accordance withtechniques herein. In 520, assume that the routine foo 502 performs acall to G1 which in turn calls G2. Thus, at a point in execution,routine foo is executing and has made a first call to routine G1 and G1has made a call to routine G2. Element 522 may represent the first callstack frame for routine foo. Element 524 may represent the second callstack frame for routine G1. Element 526 may represent the third callstack frame for routine G2.

Information stored in a stack frame (such as 522, 524, 526) for aruntime call instance or invocation may include, for example, returnaddresses, data used by that call instance for registers, variables ordata items, and the like. Elements 522 a and 524 a may denote returnaddresses, respectively, included in frame 522 for foo and frame 524 forG1. One common attack, such as may be performed by malicious code, maybe to modify return addresses such as 522 a and 524 a stored on thestack 520. Using the techniques such as described elsewhere herein forthe dynamic CFI return policy (e.g., described in connection with FIG.46 in example 500) may prevent improper or invalid returns such as usinga return address from a stack location that has been improperlymodified. However, it may be further desirable to also enforceadditional rules which provide stack protection and prevent impropermodification of stack storage locations, such as return addresses. Thus,such additional rules for a stack frame protection policy may preventmodification of 522 a or 524 a rather than allow an impropermodification of 522 a and then stop a return using the improperlymodified return address.

As described below in more detail, different levels of stack protectionmay be provided. In one aspect, stack protection may be determined basedon static procedure (also referred to as the static authority protectionmodel described elsewhere herein) or may be determined based on bothprocedure and also invocation instance of the particular procedure (alsoreferred to as the instance authority protection model describedelsewhere herein). With the static authority protection model, rules ofthe stack protection policy may provide stack protection based on theparticular procedure or routine that creates the frame. For example,rather than the stack including only a single frame for a singleinstance of foo as in 520, there may be multiple invocation instances offoo including in the current call chain at a point in time and thusmultiple call stack frames in the stack for routine foo (e.g., such asbased on recursive calls to foo). Based on the static routine orprocedure, any instance of foo may be able to modify or accessinformation in any call stack frame for an instance of foo. For example,foo instance 1 may have call stack frame 1 and foo instance 2 may havecall stack frame 2. Based on static routine or procedure for stackprotection, code of foo instance 1 may be able to access stack frames 1and 2 and code of foo instance 2 may also be able to access stack frames1 and 2. In such an embodiment, call stack frames for all instances ofthe same procedure or routine foo may be colored with the same tag. Forexample, frame 1 for foo instance 1 and frame 2 for foo instance 2 maybe both be colored with tag T1 so that rules of memory safety policywill allow the above-noted stack frame access across different instancesof the same routine or procedure.

As further finer granularity of stack protection, an embodiment may userules of the stack protection policy that further limit access of thestack based on static routine or procedure as well as the particularruntime instance of the routine or procedure (e.g., the instanceauthority protection model). For example, foo instance 1 may have callstack frame 1 and foo instance 2 may have call stack frame 2 as notedabove. Based on static routine or procedure and also invocation instancefor stack protection, code of foo instance 1 may be able to access stackframe 1 but not stack frame 2, and code of foo instance 2 may be able toaccess stack frame 2 but not stack frame 1. In such an embodiment, callstack frames for each invocation instance of a procedure or routine maybe colored with a different tag. For example, frame 1 for foo instance 1may be colored with tag T1 and frame 2 for foo instance 2 may be coloredwith tag T2 so that rules of a memory safety policy will allow theabove-noted stack frame access based on each particular invocation androutine or procedure.

An embodiment may further provide a finer level of granularity fordifferent regions or portions of the stack for a single procedure callinstance such as by coloring different objects or data items in a stackframe each with a different color (also referred to as the objectprotection model described elsewhere herein). As described elsewhereherein, the stack frame may include storage for data items or objectsused in a particular invocation of a routine or procedure where eachsuch data item or object may be tagged with a different color. Forexample, referring to FIG. 48 shown is example 530 illustrating dataitems 540 having storage allocated by a routine or procedure foo andassociated tagged memory in a stack frame 531. Element 540 denotesvariables 540 a-540 c having storage allocated in routine foo andelement 531 represents the call stack frame for this particularinvocation instance of routine foo in the call stack. Element 531includes memory region 532 for variable array 540 a, memory region 534for variable line 540 b and memory region 536 for variable password 540c. Additionally, frame 531 includes memory region 538 for stored returnaddresses. Each of the different regions 532, 534, 536 and 538 may betagged or colored with a different tag as denoted by 533. Each word inregion 532 may be tagged with Red1. Each word in region 534 may betagged with Red2. Each word in region 536 may be tagged with Red3. Eachword in region 538 may be tagged with Red4.

As yet a further variation, an embodiment may define different trustregions or boundaries for sets of code (e.g., routines, procedures,etc.) and provide different levels of protection. For example, not allroutines invoked may have the same level of trust. For example, adeveloper may have a first set of routines he/she has written and have ahigh level of trust that the operations performed by code of the firstset does not contain any malicious code. However, the first set ofroutines may make calls into a library that was provided by a thirdparty or obtained from the internet. The library may be untrusted. Thus,an embodiment may vary the level of protection based on the differentbodies of code and the particular data items used by each. For example,with reference to example 550 of FIG. 49, assume that routine foo in thetrusted user code call routine evil in the library and passes as aparameter to evil a pointer to region 534 (pointer to data item line 540b). In such a case, rather than color or tag each region of 531 with adifferent color, regions 532, 536 and 538 may all be colored with thesame color, such as Red5, and region 534 may be tagged with a differentcolor, such as Red6. This may be used to further ensure that memoryregion 534 accessed by routine evil is tagged with a different colorthan other regions of 531 as a level of memory safety since routine evilis considered untrusted code. Additionally, the pointer to region 534passed to evil may be colored or tagged with the same color Red6 as theregion 534. In this manner, memory safety policy rules may limit accessto memory used by evil to those tagged with Red6.

Whether a particular routine, library or body or code has a particularlevel of trust may be determined based on analysis using one or morecriteria and inputs. For example, based on runtime analysis and usage ofcode of a library, a level of trust may be determined. For example, ifthe library makes calls to yet other unknown or untrusted external orthird party libraries, then the level of trust may be relatively low. Alevel of trust for a body of code may be used on the source or locationfrom which the code was obtained. For example, use of code from alibrary obtained from the internet may be considered untrusted. Incontrast, code developed by a particular developer which does not invokeany untrusted code may have a high level of trust.

The foregoing and other aspects of stack frames and stack protection aredescribed in more detail below.

In connection with stack frames and with reference again to the example530, a compiler may create a new stack pointer by adding an integer (thesize of the frame) to the existing stack pointer. The old stack pointermay be pushed onto the stack (into the frame) and then recovered byreading it back from the stack. The addition to the stack pointer mayrepresent the total size of a frame that includes many independentobjects such as described above in 531 for the data items 540 a-c. Thestack needs space for these 3 data items 540 a-c and the compiler isable to determine the total space needed for the data items 540 a-c. Instandard usage, the compiler accesses storage 532, 534 and 536,respectively, for these data items 540 a-c by computing their addressesoff of the stack pointer (or frame pointer that is created from thestack pointer). Thus, the compiler, runtime, and calling conventions inan embodiment may create and use pointers to different regions of thestack call frames by doing simple pointer arithmetic.

The static authority protection model indicates authority over objectsbelongs to the static code block, such as routine or procedure thatcreates the frame. Thus, as discussed elsewhere herein, procedure foothat creates a frame has authority to create pointers to things in thatframe. In the simplest case, the same authority would allow foo toaccess any of the frames it creates, even if they were earlier or lateron the stack. Static authority means tags (e.g., colors for memorycells, colored pointers, code tags (e.g., also referred to asinstruction tags or tags on instructions) that create colored pointers)may be pre-allocated at load time. Instance authority protectionprovides authority based on the depth of the function invocation on thestack. Object protection indicates protection at the level of objectsallocated on the stack, not just stack frames. Thus, object protectionallows for detection and prevention of overflow from one object (e.g.,array, buffer) within a frame into another object on the same frame,which is something not achieved using simple stack frame granularityPUMP rules with the static authority protection model or the instanceprotection model. Object protection can be applied to both the staticauthority protection model and the instance protection model. As avariation of object protection, an embodiment may also employhierarchical object protection for hierarchical objects, such as astructure that includes multiple different data items subobjects, suchas an integer, and an array. In at least one embodiment withhierarchical objects where a first object includes one or more levelseach of one or more subobjects, a first tag may be generated for thefirst object and then additional subobject tags may be generated basedon the first tag. Each subobject tag may be used to tag a differentsubobject. The subobject tag may be a value denoting the particularposition of the subobject in the hierarchy. For example, tag T1 may begenerated for use with a structure including 2 arrays as subobjects 2and 3. A different subobject tags for each the 2 arrays may be generatedfrom T1 and used to tag the 2 array subobjects.

What will now be described is processing that may be performed inconnection with stack memory for different stack operations in anembodiment in accordance with techniques herein. At startup, the stackmemory may have all memory cells marked or tagged using a free-stackframe tag. Consistent with other discussion and techniques herein, suchtagging may be performed by invoking the PUMP rules. It should be notedthat the initial tagging of stack memory cells to the free-stack frametag may not be performed for the entire stack at once, but may rather beperformed incrementally in the kernel page fault handlers that expandthe stack.

In connection with allocating a new stack frame such as by the compiler,a new frame tag may be created for the newly allocation frame. A pointerto the new frame may be tagged with the new frame tag. For example, anembodiment may tag an instruction (e.g., such as an add instructionperforming pointer arithmetic (by adding to the stack pointer)) thatcreates a new frame pointer where the tag on the instruction triggersthe policy rule to create the new frame tag. Using rules and tagpropagation, a special tag may be created for and used to tag the stackpointer. Subsequently, for each frame pointer, a unique frame pointertag may be derived from the stack pointer special tag, and the framepointer may be tagged with its unique frame pointer tag. In such anembodiment, the frame pointer tag may be created from a tagged copy(e.g., an add or 0) of the stack pointer.

When a new stack frame is allocated such as for a new invocation of aroutine or procedure, memory cells of the newly allocated stack framemay be tagged or colored using, for example, a first technique referredto as strict object initialization or a second technique may be referredto as lazy object coloring.

With the first technique of strict object initialization, free stackframe cells of the newly allocated frame are all initially colored ortagged to the intended one or more colors such as based on the staticobjects of the frame. Such initial coloring may be performed as part ofinitialization processing of a newly allocated frame prior tosubsequently using the frame, for example, to store information for theassociated invocation. An embodiment may add code that triggers rules toperform the coloring or tagging of the free stack frame cells to theintended one or more colors such as based on the static objects of theframe. Code tags on instructions may be used to authorize and defineassociated memory cell coloring. Subsequent stores or reads of coloredmemory cells of the frame may be allowed or not based on the framememory cell color such as in accordance with memory safety policy rules(e.g., for a memory cell tagged with color C1, a rule allows a memoryoperation to access the colored memory cell contents using a pointerhaving a tag also of the same color C1 but may not allow the memoryoperation if the pointer is of a different color C2). Additionally, acode tag on an instruction may provide authority to perform the memoryoperation within a procedure.

With the second technique of lazy object coloring, there is no initialcoloring of all stack objects as with the strict object initializationtechnique. Rather, with lazy object coloring, a store to a stack memorylocation tagged as free stack frame results in triggering a rule thatallows the store and also changes the color of the memory location basedon the writer. A read to a stack memory location tagged as free stackframe is an uninitialized memory read and may be allowed/not alloweddepending on whether the policy allows/disallows uninitialized memoryreads. With lazy object coloring, no initial block of code is executedthat invokes rules to completely initially tag all memory cells of aframe upon creation. Rather, memory cells are tagged by rules invoked inconnection with store operations.

In at least one embodiment, whether to use strict object initializationor lazy object coloring may depend on a desired level of protection andan occurrence of untenable vulnerabilities.

Code within a routine or procedure that directly accesses data from thestack/frame pointer is code tagged to allow it do so. In connection withlazy object coloring, storing to a memory cell results in coloring thememory cell based on the writer as noted above. For example, withreference back to the example 530, a store instruction of routine foohaving frame 531 may write a value to a memory location in array 532. Inaccordance with a current stack protection policy in effect, in orderfor a store instruction to write to a location in array 532 of the callframe for foo, the store instruction may be required to have a tag ofRed1. A first rule of the policy may be triggered to perform this checkfor a store instruction. Thus, an embodiment may have a compilergenerate a code sequence that triggers the first rule to tag the storeinstruction with Red1. (e.g., As a variation to the foregoing, the tagon a memory cell, such as Red1, may be related to but not the same asthe tag on the instruction store or other instruction. For example, the“Red1 code” CI tag may indicate that the instruction having this tag canaccess Red1 tagged memory cells and may create Red1 tagged memorycells). When the store instruction is the current instruction, theforegoing first rule may be triggered which checks the instruction tagto ensure it is Red1. As an output, the rule may tag the memory locationin array 532 with the Red1 tag.

Code within a procedure that creates a pointer to a particular object istagged to taint or set the pointer for that object. The pointer may befor the procedure's own use in subsequent instructions and/or may bepassed to another procedure as an argument.

Storing register values to a frame or restoring register values from aframe may be based on the frame authority. The memory location(s) of thestack frame storing the register values may be treated as a uniqueobject in the stack frame. Instruction tagging provides authority forsuch tagged store and load instructions. With lazy-object-coloring, thestore instruction tagged with the authority to store data to a memorycell also provides the authority to tag the memory cell based on thewriter (e.g., procedure including the store instruction).

Procedure arguments passed on the stack may be marked with a tag thatallows both the caller and callee to access. Note that return addressesmay be specially tagged (e.g., the dynamic CFI return policy describedelsewhere herein such as in connection with FIG. 46). Thus, if a returnaddress is stored on the stack (e.g., such as in connection with nestedor recursive calls), stores will not be allowed to overwrite returnaddresses on the stack due to the tagging on the return addresses). Whena stack-derived pointer is passed to another frame in connection with acall to another procedure, memory accesses performed using the pointerresult in triggering rules of a memory safety policy as describedelsewhere herein. The instruction that created the pointer to a memorylocation may be tagged based on the particular memory location's tag.The instruction tag may indicate authority to access the memorylocation. The instruction may trigger a rule that tags the pointer todenote authority to access the memory location. For example, the rulemay assign the pointer the same tag as the instruction or a variationbased on the instruction tag. Thus, in one aspect, the instruction thatcreated the pointer is also creating a capability to access the memorylocation through the pointer and sharing that capability through thepointer passed as an argument to the called procedure. It should benoted that with lazy-object-coloring, the pointer will need to have atag providing authority to tag free-stack-frame-cells, which may not beallowed on heap memory safety pointers.

In connection with a return or other operation resulting in removing aframe from the stack (e.g., such as due to completion of a calledroutine), tagged code may clear the frame. The tags on such codeprovides the authority to change any frame object tags associated withthis frame to the free-stack-frame-cell tag.

Code of a program executed in an embodiment of a computer system inaccordance with techniques herein may include code that performsexception handling. As known in the art, exception handling isprocessing performed responsive to an exception denoting an occurrenceof an anomalous or exceptional condition requiring special processingperformed by the exception handler. Thus, when an exception occurs at afirst point in a program, the normal flow of program execution may beinterrupted so that control is transferred to an exception handler.Prior to transferring control to the handler, the current state ofexecution may be saved in a predetermined location. If program executionmay be resumed after the exception has been processed by the handler,execution of the program may resume (e.g., control may then betransferred back following the first point in the program). For example,a divide by zero operation may result in an exception that iscontinuable where the program execution may resume after the exceptionis handled by the handler. In connection with implementing an exceptionhandler, an embodiment may use library routines such as setjump andlongjump. For example, setjump and longjump may be standard C libraryroutines, respectively, setjmp and longjmp, defined as follows:

-   -   int setjmp(jmp_buf env)        where setjmp sets up the local jmp_buf buffer and initializes it        for the jump. Setjmp saves the program's calling environment in        the environment buffer specified by the env argument for later        use by longjmp. If the return is from a direct invocation,        setjmp returns 0. If the return is from a call to longjmp,        setjmp returns a nonzero value.    -   void longjmp(jmp_buf env, int value)        where longjmp restores the context of the environment buffer env        that was saved by invocation of the setjmp routine in the same        invocation of the program. Invoking longjmp from a nested signal        handler is undefined. The value specified by value is passed        from longjmp to setjmp. After longjmp is completed, program        execution continues as if the corresponding invocation of setjmp        had just returned. If the value passed to longjmp is 0, setjmp        will behave as if it had returned 1; otherwise, it will behave        as if it had returned value.

Thus, setjmp may be used to save a current state of a program. The stateof a program depends on, for example, the contents of memory (i.e. thecode, globals, heap, and stack), and the contents of its registers. Thecontents of the registers includes the stack pointer, frame pointer andprogram counter. Setjmp saves the current state of the program so thatlongmp may restore the program state and thus return the state of theprogram execution to what it was when setjmp was called. In other words,longjmp( ) doesn't return. Rather, when longjmp is invoked, executionreturns or resume to the particular point denoted by the previouslysaved program state (as saved by setjmp). Thus, longjmp( ) may be usedto transfer control from a signal handler back to a saved executionpoint in a program without using standard calling or return conventions.

For example, reference is made to FIG. 50. In the example 560, routinemain 562 may call routine first 563, and routine first 563 may callroutine second 564. As illustrated, main 562 may include a call tosetjmp at point X1 prior to calling routine first. The first time setjmpis called at point X1, it returns a zero and then routine first iscalled. After longjmp is executed, setjmp returns 1. Routine second 564includes a call to longjmp at point X2 which causes a transfer ofcontrol back to main at location X1 where setjmp was called. Setjmp isnow called again and returns a 1 so first is not called and the controlproceeds to NEXT.

In connection with a stack protection policy, it may be desirable toclear the stack prior to resuming execution to the point X1 previouslysaved by settmp. For example, based on the above call chainmain-first-second, 3 stack frames may exist in the call stack andprocessing may be performed to clear stack memory associated withinvocations in the call chain between the longjmp call and setjmp call.In particular, code of longjmp may include code that clears stack framesfor first 563 and second 564 in this example. What will now be describedare techniques that may be used in connection with performing such stackclearing in accordance with a stack protection policy.

In connection with a stack protection policy when performing a setjmpthat saves program state to stack memory, an embodiment may tag thecurrent stack pointer memory cell with a distinguished tag component sothat, in connection with a subsequent longjmp, rules may check that thestack hasn't changed since the setjmp. Data may be saved to the setjmpdata structure, jmpbuf, denoting the current program state. The saveddata may include the stack pointer, program counter, a first pointer(tagged as being a pointer that is allowed to point to a memory locationtagged with the distinguished tag component (e.g. point to the currentstack pointer memory cell), and a second pointer (tagged aslongjmp-clearing-authority-pointer to provide authority to performlongjmp processing). In at least one embodiment, thelongjmp-clearing-authority-pointer may only provide authority to cleartags associated with frames in the set of procedures that could berecursively called from this procedure.

In connection with a stack protection policy when performing a longjmp,code may check that the current stack pointer denotes a deeper stackposition than the saved stack pointer of the set jump structure (e.g.,setjmp data structure, jmpbuf). A rule may be triggered that checks thatthe memory cell of the set jump structure containing the saved stackpointer (as saved by setjmp) has a tag that is compatible with thetagged first pointer (of the set jump structure). Code may be executedthat clears all stack memory locations between the current stack pointerand the saved stack pointer (as previously saved by set jump in the setjump structure). Such code may be perform the clearing using the secondpointer noted above that is tagged as thelongjmp-clearing-authority-pointer providing the stack clearingauthority (e.g., second pointer used to point to stack locationscleared). Rules may be triggered by the code performing the clearingwhere the rules check that the second pointer is tagged aslongjmp-clearing-authority-pointer. Instructions in longjmp are uniquelytagged so that invoked rules allow the uniquely tagged instructions touse of a pointer tagged as longjmp-clearing-authority-pointer. Othercode that is not in longjmp cannot use a pointer tagged aslongjmp-clearing-authority-pointer (e.g., the other code is not taggedto allow use of longjmp-clearing-authority-pointer).

In at least one embodiment, tagging of instructions may be performed byhaving the compiler generate an instruction sequence that invokes rulesto perform desired instruction tagging and/or memory location tagging.For example, for stack memory location tagging, the compiler maygenerate an instruction sequence with store instructions that triggerrules to initialize or reset the tag of a stack location. For tagginginstructions, the compiler may generate an instruction sequence withstore instructions that trigger rules to tag an instruction where thetag for the instruction may be based on the color associated with atagged memory location accessed by the instruction. In connection with areturn from a call having an associated stack frame, code may be addedthat clears the frame from the stack. When strict object initializationis employed and a new frame created in response to a call, code may beadded that appropriately tags or colors objects of the new frame.

What will now be described with reference to FIGS. 51-53 are examples ofdifferent unauthorized or unintended modifications that may be made tothe stack (“stack attacks” referring to attacks made through stackmodifications) such as, for example, made by malicious code orunintended stack modifications by non-malicious code (e.g., accidentaloverwrites or buffer overflows).

FIGS. 51-52 illustrate actions that may be taken to prevent stackattacks in connection with stack modifications made by a code modulesuch as third party code (e.g., library routine invoked) and may becharacterized as an arbitrary attacker model. Thus, the cases in 570 and575 may occur, for example, as a result of a called third party libraryroutine including code that performs the unauthorized or unintendedstack modification. Additionally, the stack modification may also bemade by yet another routine further invoked by code of the calledlibrary routine. Each line of 570 and 575 includes 3 columns ofinformation. For each of lines 572 a-h, column 1 identifies an item toprevent denoting undesired runtime execution behavior, column 2identifies a preventive action that may be taken to avoid the undesiredbehavior of column 1, and column 3 identifies one or more mechanismsthat may be used to implement or enforce the preventive action of column2. Generally, in column 3, alternate mechanisms are listed which may beeach be implemented independently and separately depending on theparticular system. For example, a conventional system may use separateprocesses as a first mechanism while a second system may alternativelyuse a capability and a second system may alternatively use coloring ortagging of the particular stack locations.

To further illustrate and consistent with discussion elsewhere herein,code, such as prolog code executed when a call is made, writes returnaddresses and registers to the stack. The prolog code may invoke rulesthat tag the stack locations with special tags to limit what code canmodify or generally access the stack locations. For example, prolog codemay perform memory writes/stores to store return addresses, registers,and the like, in a memory cell of a stack frame. Such write/storeinstructions of the prolog code may invoke a rule that tags a memorycell of the stack frame with a special tag STACK FRAME TAG to mark thememory location as special and limit what code can modify the memorycell. The write/store instructions of the prolog code may also be taggedwith PROLOG STACK TAG to limit the instructions that can perform thistagging. The following is an example of logic enforced by the ruleinvoked by the write/store instructions of the prolog code that tags amemory cell of the stack frame with a special tag STACK FRAME TAG tomark the memory location as special and limit what code can modify thememory cell:

-   -   If (CI=PROLOG STACK TAG) AND (this is a memory write operation)        then output or Rtag=STACK FRAME TAG        In the foregoing rule logic, output tag refers to the tag placed        on the stack location.

In a similar manner, other code, such as epilogue code invoked withperforming a return, may be allowed to clear the stack, or portionthereof. The epilogue code may be tagged with the special tag of EPILOGSTACK TAG (e.g., CI tag) and may be given authority through access of apointer tagged with the special tag STACK FRAME TAG.

The epilogue code may perform the foregoing stack clearing usingwrite/store operations using the pointer specially tagged with STACKFRAME TAG. To further limit performing stack clearing, the epilogue codemay be tagged as noted above. In such an embodiment, the write/storeinstructions may invoke a rule implementing the following logic toenforce the policy where stack clearing may only be performed byepilogue code using the specially tagged pointer (tagged with STACKFRAME TAG):

-   -   if (CI=EPILOG STACK TAG) AND (memory write operation) AND        (Mtag=STACK FRAME TAG) then output or Rtag=Default tag

Code that is intended to restore return addresses and registers from thestack may be given authority to read these specially tagged memory cellsof the stack. Such authority may be given, for example, by any of:tagging the code (CI tag) to denote the code is allowed to access thespecially tagged memory cells of the stack, tagging the PC to indicatethe code has the authority, or tagging a pointer used by the code wherethe pointer points to the specially tagged memory cells and the tag onthe pointer denotes the access authority. For example, a read/loadinstruction may be given authority to read the stack memory cells taggedwith STACK FRAME TAG. In one embodiment, the read/load instruction maybe given authority by allowing only read/load instructions using thespecially tagged pointer (tagged with STACK FRAME TAG) to read from astack memory location. Rule logic allowing only read/load instructionsusing the specially tagged pointer (tagged with STACK FRAME POINTER) toread from the specially tagged stack memory location (tagged with STACKFRAME TAG) may be:

-   -   if (memory read operation) AND (R2tag=STACK FRAME POINTER) AND        (Mtag=STACK FRAME TAG) then Rtag=DEFAULT TAG

As a variation to the foregoing, the read/load instruction may be givenauthority by tagging a pointer used by the read/load instructions withthe special tag STACK FRAME TAG.

Rule logic allowing only read/load instructions using the speciallytagged instructions (tagged with STACK FRAME INSTRUCTION) to read from astack memory location may be:

-   -   if (memory read operation) AND (Cltag=STACK FRAME INSTRUCTION)        AND (Mtag=STACK FRAME TAG) then Rtag=DEFAULT TAG)

Examples of mechanisms are described below and elsewhere in more detail.

Element 572 a identifies an undesired runtime behavior of a calledroutine (callee) that never returns to the calling routine (caller). Toprevent this behavior, an action taken may be to have a timeoutassociated with each call where a maximum amount of time may be allowedto complete the invoked routine. After the maximum amount of timeelapses, runtime execution of the invoked routine is terminated.Mechanisms to implement the timeout may include having the invokedroutine of the third party code be made from a separate thread thatenforces the timeout, or directly limiting the amount of time of thecalled routine using a time or instruction limited call.

Element 572 b identifies an undesired runtime behavior of resourceexhaustion where the called routine may use up an available resource,such as memory. To prevent this behavior, an action taken may be tolimit resource made available to the called routine. Mechanisms toimplement the timeout may include having the invoked routine of thethird party code be made from a separate thread that enforces themaximum resource limits, or directly limiting the amount of resource ofthe called routine using a special instruction limited call.

Element 572 c identifies an undesired runtime behavior of the invokedroutine exercising unexpected authority such as by making an expectedcall to yet another routine. To prevent this behavior, an action takenmay be to limit the authority of the called routine to the minimumprivilege allowable. Mechanisms to implement this may include taggingthe PC with the authority and control capabilities of the callee orcalled routine and limiting the portion of the file system or otherresources accessible to the called routine, and limiting the allowablesystem calls the invoked routine can make.

Element 572 d identifies an undesired runtime behavior of the calledroutine reading items left in registers by other routines subsequentlycalled by the called routine (e.g., mycode calls P1 in the library andP1 further calls routine evil and P1 may read data left in registers byevil). To prevent this behavior, an action may be taken to clear thenon-input and non-return registers. Mechanisms to implement this mayinclude performing explicit register clearing, coloring portions of thestack including the non-return and non-input registers so that theycannot be read by the called routine, and having a separate processinvoke the called routine.

Element 572 e identifies an undesired runtime behavior of the calledroutine reading items left on the stack by other routines subsequentlycalled by the called routine (e.g., mycode calls P1 in the library andP1 further calls routine evil and P1 may read data left on stack byevil). To prevent this behavior, an action may be taken to make thecalled stack inaccessible (e.g., stack region used by the furtherinvoked other routines such as evil are inaccessible to the first calledroutine such as P1). Mechanisms to implement this may include usingseparate stacks (e.g., for the first called routine P1 and the furtherinvoked routine evil), capabilities (e.g., tag PC or use speciallytagged pointer allowed to access particular stack regions to limitability or authority of code allowed to read stack areas), coloring(e.g., tag data areas of stack to limit what code can access), andhaving a separate process invoke the called routine.

Element 572 f identifies an undesired runtime behavior of the calledroutine writing over items in the stack prefix (e.g., overwriting thereturn address area identifying the return address). The stack prefixmay be an area of stack that includes information needed to return tosome prior caller. To prevent this behavior, an action taken to make thestack prefix inaccessible or unwritable to the called routine.Mechanisms to implement this may include having the called routine andthe user code invoking the called routine use separate stacks, usingcapabilities (e.g., allow access through specially tagged code or codeprovided authority through PC tag or specially tagged pointer), usingcoloring (e.g., tagging the data items of the stack prefix with specialtags so that called routine is not allowed to access), and having aseparate process invoke the called routine.

Element 572 g identifies an undesired runtime behavior of the calledroutine read data in the stack prefix. To prevent this behavior, anaction taken to make the stack prefix inaccessible to the called routineusing mechanisms similar to those described with 572 f.

Element 572 h identifies an undesired runtime behavior of the calledroutine redirecting control flow in the stack prefix such as byoverwriting the pointer to the return address where the pointer isstored in the stack prefix. To prevent this behavior, action may betaken to protect the return address stored in the stack prefix. In oneaspect, element 572 h identifies a particular instance of 572 h and thusthe mechanisms of 572 h are similar to those of 572 f. Mechanisms toimplement this may include having the called routine and the user codeinvoking the called routine use separate stacks, using capabilities(e.g., allow access through specially tagged code or code is providedauthority through PC tag or specially tagged return pointer that istagged by access authority), using coloring (e.g., tagging the memorylocation of the stack prefix including the return address with a specialtags so that called routine is not allowed to access), and having aseparate process invoke the called routine.

FIG. 53 illustrates actions that may be taken to prevent stack attacksin connection with an arbitrary input attacker model.

Element 581 a identifies an undesired runtime behavior of executing codewriting over unintended items in the current frame of the executingroutine. To prevent this behavior, an action taken may be to maintainobject integrity. Mechanisms to implement this may include usingcapabilities by object (e.g., allow access through capability providedwith specially tagged code or code provided authority through PC tag orspecially tagged return pointer that is tagged by access authority), orcolor by object (e.g., tagging the memory locations of an object).

Element 581 b identifies an undesired runtime behavior of reading itemsin the current frame of the executing routine. To prevent this behavior,an action taken may be to maintain object integrity. Mechanisms toimplement this may include using capabilities by object (e.g., allowaccess through capability provided with specially tagged code or codeprovided authority through PC tag or specially tagged return pointerthat is tagged by access authority), or color by object (e.g., taggingthe memory locations of an object with object specific tag).

Element 581 c identifies an undesired runtime behavior of executing code(having current frame) writing over unintended items in the predecessorframe (e.g., of other routine that invoked the executing code). Toprevent this behavior, an action taken may be to isolate or separatetack frames. Mechanisms to implement this may include using capabilitiesby frame (e.g., allow access through capability provided with speciallytagged code or code provided authority through PC tag or speciallytagged return pointer that is tagged by access authority), or color byframe (e.g., tagging the memory locations of frame with frame-specifictag).

Element 581 d identifies an undesired runtime behavior of executing code(having current frame) reading items in the predecessor frame (e.g., ofother routine that invoked the executing code). To prevent thisbehavior, an action taken may be to isolate or separate tack frames.Mechanisms to implement this may include using capabilities by frame orcolor by frame as described with element 581 c.

Element 581 e identifies an undesired runtime behavior of executing code(having current frame) reading items left of the stack by anotherroutine invoked by the currently executing code. The preventive actionis to make the called stack of the invoked routine inaccessible to thecurrently executing code. Mechanisms to implement this may include usinga separate process, a separate stack, capabilities and coloring in amanner similar to as described in connection with 572 g.

Element 581 f identifies an undesired runtime behavior of executing code(having current frame) modifying the return pointer (e.g., location instack including return address in routine that invoked executing code).The preventive action is to protect the return pointer or location inthe stack including the return address. Mechanisms to implement this mayinclude using capabilities and coloring in a manner similar to asdescribed in connection with 572 g.

An embodiment in accordance with techniques herein may use the PUMP rulemetadata processing system as part of another hybrid system to learn andvalidate new set of rules. For example, the PUMP rule metadataprocessing system may be used to learn (e.g., through logging) allowedcontrol flow and thus determine rules and allowed valid controltransfers for an executing program. The rules and allowed valid controltransfers may be then be used as the rules and set of valid controltransfers of a CFI policy enforced for the program that was executed.

To further illustrate learning rules and control transfers for theprogram's CFI policy, a first training or learning phase may beperformed. In this first phase, the program is executed with all controlpoints (e.g., branch or transfer source and targets) tagged and atraining version of a CFI policy where there are no rules for controltransfer instructions. Thus, each time there is a control transfer, suchas a branch or jump instruction, there is a PUMP rule cache miss causingtransfer of control to the cache miss handler of the PUMP rule metadatasystem. The cache miss handler may perform processing to log informationregarding the control transfer. The information logged may include, forexample, the source location of the transfer and the target location ofthe transfer. Other information may also include, for example, thecalling procedure or routine from which the transfer is made (e.g., andincludes the source location) and the called procedure or routine towhich control is transferred (e.g., and includes the target location).More specifically, in the learning or training phase, the first time aparticular transfer of control occurs, the cache miss handler computes anew rule of the learned set of rules for that particular transfer ofcontrol from a source to a target. Subsequent runtime transfers ofcontrol from the same source to the same target use this computed rule.In this manner, if the program is presumed bug-free and a non-attackprogram (not malicious code) and all control paths are exercised duringa program run, the logged set of control transfers, as indicated by thelearned rule set at the end of program execution, represents as allvalid or allowable control transfers for this particular program. Thus,the learned set of rules may denote an initial or first set of rules forthe CFI policy for the program.

Processing may be performed to validate the learned set of rulesdenoting the CFI policy for the program. The validation may includeensuring that none of these rules allow invalid control transfers. Thevalidation of the learned set of rules may be performed in any suitablemanner. For example, an embodiment may run an analysis tool thatvalidates each rule. The tool may, for example, examine the binary orobject code, symbol table and original source code, and the like, tovalidate that each rule corresponds to an allowed transfer. To furtherillustrate, validation may examine the binary code that has all controlpoints (e.g., branch or transfer source and targets) tagged. In thismanner, the tagged binary or source code denotes the valid set of allpotential source and target locations thereby providing a valid set ofpotential source and targets that can actually be used in a runtimetransfer of control. Any runtime transfer of control logged should onlyoccur from a source to a target where each of the source and target areincluded in the valid set. For example, the tagged binary or source codemay include locations A1, A2 A3 and A4. Any logged transfer of controlshould include a source that is A1, A2, A3 or A4, and a target that isA1, A2, A3 or A4. If a logged runtime control transfer denoted by afirst rule is from A1 to B7, the first rule may be invalidated since B7should not be a target of a control transfer (e.g., B7 is not includedin the set of statically determined possible control points taggedconsistent of A1, A2 A3 and A4). In one aspect, the learned set of rulesmay be characterized as a candidate set of rules which may be furtherreduced via rule removal as a result of validation processing.

All rules of the initial or learned set of rules for the CFI policy forthe program that have been validated may then be used as a validated setof rules included in a CFI policy that is then enforced for the program.

Referring to FIG. 54, shown is an example summarizing processing justdescribed as may be performed by an embodiment in accordance withtechniques herein for learning, validating and using policy rules. In602, the program may be initially executed with no CFI policy rules ineffect so that each new transfer of control causes a rule cache miss andtriggers the cache miss handler to generate a new rule regarding thetransfer of control encountered at runtime. The new rule may identify atransfer of control from the source and target and may be included in afirst set of learned rules 604. At the end of program execution, thefirst set of learned rules 604 includes a rule for each differenttransfer of control that occurred at runtime. The first set of learnedrules may then be validated in processing of 606 to ensure each rulerepresents a valid control transfer. Processing of 606 may use a tool asdescribed above for automated rule validation and may also include otherprocessing. For example, validation processing of 606 may includepresenting a rule that has been validated by the tool to a user forfurther confirmation that the control transfer is valid. The second setof validate rules 608 may be generated as a result of rule validationprocessing 606. Subsequently, the second set of validated rules 608 maybe used by the PUMP system as the CFI policy enforced when executing theprogram at a second point in time in 610.

Thus, for foregoing first program execution in 602 may be used todetermine a set of valid control transfers for the program. However, itmay not be reasonable to assume that this single program executionexercises all control paths whereby the control transfers identified in608 as valid may denote less than all possible valid control transfers.In this case, processing may be performed as described above inconnection with 610 using the validated set of CFI policy rules. Duringruntime, if a control transfer is encountered causing a rule cache miss(e.g., indicating an unforeseen control transfer not having a rule in608), additional checking may be performed at runtime, for example, tovalidate the control transfer such as described above (e.g., using theset of possible control points tagged in binary code or annotated in asource program). If the control transfer is determined as invalid, afault or exception may be triggered.

As an alternative, if a control transfer is encountered causing a rulecache miss thereby denoting an unexpected runtime control transfer, thecache miss handler may record the unexpected transfer rule for latervalidation and also allow the unexpected transfer of control to proceedwith additional or different policies in effect. For example, for anunvalidated control transfer, the transfer may be considered untrustedso policies may be modified to reflect a higher level of protection dueto the untrusted nature of the unvalidated control transfer. Forexample, the unexpected transfer may transfer control to a libraryroutine. The library routine may be executed using policies reflecting ahigher level of protection and less trust than those in effect prior tothe unexpected transfer. For validated control transfers, a first stackprotection policy may be in effect at a first point in time prior to theunexpected transfer of control and a second stack protection policy ineffect after the unexpected transfer of control. The first stackprotection policy may enforce static procedure authority. The firstprotection policy may not include any coloring at the object level asdescribed elsewhere herein with the object protection model. After theunexpected control transfer, the second stack protection policy ineffect may provide for stack protection in accordance with the objectprotection model described elsewhere herein with strict object coloring.Thus, code executed once the unexpected control transfer is encounteredmay utilize the more restrictive second stack protection policyproviding a tighter finer level of granularity of stack protection.Additionally, the program execution may continue with a reduced level ofpriority once the unexpected transfer of control occurs.

Referring to FIGS. 55 and 56, shown are flowcharts 620, 630 ofprocessing steps that may be performed in an embodiment in accordancewith techniques herein using a set of validated rules, such as the rulesof a CFI policy for a program described above. Flowchart 620 describes afirst set of processing steps that may be performed in connection withan unexpected transfer of control not having a rule in the CFI policy ofan executing program. Flowchart 631 describes a second set of processingsteps that may be performed in connection with an unexpected transfer ofcontrol not having a rule in the CFI policy of an executing program.

Referring to flowchart 620, at step 622 a program may be executed usinga set of validated rules. At step 624 during program execution, aruntime transfer of control is performed. At step 626, it is determinedwhether there is a rule cache miss thereby indicating that the transferis unexpected. In particular, if there is a rule in the second set ofvalidated rules for the runtime transfer of control, then the transferof control is expected where step 626 evaluates to no and processingcontinues with step 628 where the control transfer is performed and theprogram continues execution.

If step 626 evaluates to yes (e.g., cache miss indicating an unexpectedtransfer of control), processing continues with step 632 where runtimevalidation processing is performed for the unexpected control transfer.In particular, the miss handler may perform processing that attempts tovalidate the unexpected transfer. Examples of rule validation processingmay include determining whether the runtime source and target locationsare included in a set of potential control transfer points as describedabove that may be determined using tagged binary code, the originalsource program and symbol table, and the like. At step 634, it isdetermined whether the validation processing of step 632 determined theunexpected transfer of control is valid. If step 634 evaluates to yes,processing continues with step 636 where the new rule is added to thesecond set used as the CFI policy for the program and processing of theprogram continues. If step 634 evaluates to no, program execution may beterminated, for example, by causing a trap.

Referring to flowchart 631, steps 622, 624, 628 and 628 are as describedabove in connection with flowchart 620. If step 626 evaluates to yes,control proceeds to step 639 where the unexpected transfer of controlmay be recorded (e.g., candidate rule for unexpected transfer of controlrecorded) for later validation. In step 639, the program is allowed tocontinue execution even when the transfer of control is unexpected.However, in step 639, program execution continues, for example, using aset of one or more restrictive policies, reduced execution priority, andthe like, such as noted above.

The processing described above of such as described above may besimilarly performed in connection with other policies such as tainttracking. For example, for taint tracking, a first learning or trainingphase may be performed to learn rules of a policy via program executionby having the cache miss handler “log” each cache miss. As describedherein, taint tracking may include tagging data based on the code thatproduces or accesses it (e.g., such as using the CI as describedelsewhere herein.) One reason to taint data based on code or source isto make sure that programs are properly contained and do not performunwanted or improper data accesses. For example, rules may be used toassure that data tainted by the JPEG decoder never flows into thepassword database, or that credit card data, social security number orother personal information is only accessed by a particular set of oneor more restricted applications. With determining a taint trackingpolicy, processing may be performed for a learning or training phasewith no taint tracking rules run on test data that causes a cachehandler miss the first time it sees a particular flow of data (e.g.,which routines of a program access what data, what user input is writtento what database, and the like) and records the rule. In a mannersimilar to that as described above for the CFI policy, at the end of atest run of the first learning phase, there is a set of learned rules toapply to protect the program during operation. Validation processing ofthe learned set of rules may be also be performed using a tool or othersuitable means as noted above for the CFI learned set of rules. Suchvalidation processing for taint tracking may include ensuring that eachdata flow or access is proper.

Also, in a manner similar to that as described in connection withflowcharts 620 and 631, the validated set of rules may be used with thePUMP system where a cache miss handler handles processing for any dataaccess that does not have a corresponding rule in the validated set.Similar to processing of flowchart 620, the cache miss handler may thenalso perform runtime validation processing (e.g., similar to step 632)to determine whether a candidate rule for the data access or data flowis valid and allow program execution to continue (e.g., similar to steps634, 636) or not (e.g., similar to step 638). Alternatively, similar toprocessing of flowchart 631, the cache miss handler may record acandidate rule for the unexpected data access or data flow that may bevalidated offline (e.g., not during runtime) and continue programexecution using, for example, more restrictive policies, reducedpriority, and the like (e.g., similar to step 639).

The examples above describe a generally binary learning process. Anembodiment in accordance with techniques herein may further support useof statistics in making a decision about whether or not to allow anevent (e.g., control transfer or data access). In at least oneembodiment, a counter may be added to each rule to count the number ofuses of each rule during program execution. When the rule is evictedfrom the PUMP cache, processing may add the accumulated rule usage intoa global, software count that may be used to provide additionalstatistics regarding rule usage. The count may also be used to allowsomething to occur a limited number of times. For example, in connectionwith taint tracking rules tracking the flow of data from a source to atarget, a limited threshold amount of data may be allowed for unexpecteddata flows between a source and target (e.g., X amount of data read froma particular database by a particular program). Once that thresholdamount has been transferred, no additional data may be transferredbetween the source and target until the corresponding candidate rule hasbeen successfully validated. With the limited use case with thethreshold amount, the PUMP system (e.g., miss handler) may allow aninstruction lacking a rule to occur some limited number of times.Aggregation or counting as applied to the threshold may be done indifferent ways. For example, consider unexpected control transfers.Non-aggregated, the cache miss handler may not allow the same unexpectedcontrol transfer without a validated rule to occur more than 5 times.Aggregated, such as across all unexpected control transfers for aprogram, the program may be allowed to make a maximum number of 100unexpected control transfers. This may be useful for example, for caseswhere it is acceptable for a single instance of an unexpected transferof control or unexpected data access to occur. For example, a singlequery to examine data from a particular source may be allowed. However,if above a threshold number of queries are performed to the data source(e.g., particular database), the program should be flagged or stopped.

The more general statistics case may be used for learning the range ofnormal behavior. For example, a program may be executed in a learningphase to determine a relative usage of the different rules of a policy(e.g., ratio of usage of each rule). For example, relative usage of eachrule invoked for a runtime control transfer may be recorded. Ideally,such execution may be performed for the program using many differentdata sets to learn what may be considered average or normal programbehavior. Rule learning and validation may then result in a set of rulesfor the validated control transfers (as described above) andadditionally a ratio indicating a relative usage of each validated rule.Both the validated rules and associated usage ratios may be used duringsubsequent processing as the enforced policy rules. During subsequentprogram execution when the policy is enforced, the PUMP system may checkif current rule usage is out of line with the expected ratio. Anembodiment may include, for example, a range or maximum expected usagefor the rule where control transfers invoking the rule more than themaximum may be flagged. For example, the program invoking the particularcontrol transfer rule more than the maximum may be flagged for furtherinspection or analysis. Using this mechanism, program runtime behaviorcan be monitored similar to the way network behavior is monitored togenerate firewall rules. Statistical learning algorithms can be used tocapture rule usage, and perhaps other standard runtime characteristicslike main-memory traffic and cache miss rates, to learn normal caseversus attack behavior. In embodiment applying the limited-use thresholdas described above, if a program exhibits other runtime behavior that isanomalous or otherwise may be considered untrusted, the use limits maybe greatly reduced or otherwise set to zero. Alternatively, if a programexhibits normal runtime behavior or otherwise may be considered trusted,the use limits may be set much higher or increased in comparison theuntrusted scenario.

The techniques above may be used to determine a set of valid rules of apolicy, such as valid control transfers reflected in rules of a CFIpolicy, without having a compiler output any additional information.Thus, an embodiment in accordance with techniques herein may have twoversion of each policy—one used for the learning phase and another usedfor subsequent enforcement. The learning phase may be used as anautomated diagnostic mode to discover allowable data accesses or flowsfor taint tracking, discover control transfers for a CFI policy, and thelike.

What will now be described are examples of architectures that may beused in embodiments in accordance with techniques herein using a RISC-Vprocessor. Additionally, described below are techniques that may be usedin connection with performing processor-based mediated data transfersbetween untagged and tagged data sources used by the processor. Suchtechniques provide for tagging external untrusted data that may bebrought into the system for use by the processor and also removing tagsfrom tagged data used within the system to generate untagged data foruse outside of the system.

Referring to FIG. 57, shown is an example of components that may be usedin an embodiment in accordance with techniques herein to mediate betweentagged and untagged data. The example 700 includes a RISC-V CPU 702,PUMP 704, L1 data cache 706, L1 instruction cache 708, interconnectfabric 710 used internally within the system for tagged data transfers,boot ROM 712 a, DRAM controller (ctrl) 712 b, and external DRAM 712 cstoring tagged data. Also included are add tag 714 a and validate droptag 714 b which are hardware components, interconnect fabric 715 usedfor transferring external untagged data in from untagged memory 716 foruse by the processor 702 and transferring untagged data out to theuntagged memory 716. It should be noted that other sources 701 ofexternal untagged data, besides untagged memory 716, may be connected tothe untagged fabric 715. For example, element 701 may include untaggeddata stored in flash memory, untagged data accessible from the network,and the like. The DRAM controller (ctrl) 712 b is a controller used forreading data from and writing data to DRAM 712 c. Boot ROM 712 a mayinclude boot code used when booting the system.

The example 700 illustrates a separate tagged fabric 710 and untaggedfabric with the processor 702 used for moving data between the two. Addtag 714 a takes as an input untagged data and tags it with a tagindicating that the data is public (can be used outside the systemdescribed herein) and untrusted (since the source may be unknown orotherwise not from a known trusted source). In at least one embodiment,the untagged data of untagged memory 716 may be received by 714 a. Thereceived untagged data from 716 may be encrypted whereby add tag 714 asimply adds an untrusted tag to the received encrypted data. Thereceived data may be encrypted using asymmetric cryptography, such aspublic key cryptography using a public-private key pair, or othersuitable encryption technique known in the art. The received data may bestored in an encrypted form. As known in the art, for a public-privatekey pair of an owner, the private key is known only to the owner but thepublic key is made public and used by others. A third party may encryptinformation sent to the owner using the owner's public key. The ownermay then decrypt the received encrypted information using its privatekey (not shared with anyone else). In a similar manner, the owner mayencrypt information using his private key where the encryptedinformation is sent to a third party that decrypts the encryptedinformation using the owner's public key.

Validate drop tag 714 b may receive tagged encrypted data and remove thetag thereby resulting in untagged encrypted data that is being exportedto the untagged memory 716. Such untagged encrypted data stored inmemory 716 may be used, for example, on another system and processor notusing tags and associated metadata rule processing as performing usingthe PUMP described herein.

In at least one embodiment, the untagged data received at 714 a may beencrypted, as noted above, and also signed to provide integrity of thedata. Furthermore, signatures may be used in validating the receiveddata item to ensure authentication and data integrity (e.g., has notbeen modified since sent by the original sender that signed, ensure datawas sent by the sender signing the data). For example, an owner may hasha message to produce a hash value or “digest,” and then encrypt thedigest with the owner's private key to produce a digital signature. Theowner may send the message and signature to a third party. The thirdparty may validate the received data using the signature. First, thethird party may decrypt the message using the owner's public key. Thesignature may be verified by computing the hash or digest of thedecrypted message, decrypting the signature with the owner's/signer'spublic key to obtain an expected digest or hash, and comparing thecomputed digest with the decrypted expected digest or hash. Matchingdigests confirms the message has not been modified since it was signedby the owner.

In operation, an instruction, such as a load instruction, may referencedata stored in the untagged memory 716 which is then transferring intothe data cache 706 for use in instruction execution. For such a loadinstruction, the data may be transferred from 716 over 715 forprocessing by 714 a which outputs the tagged data (tagged as untrustedand public). The tagged data output by 714 a is stored in the L1 datacache 706 for processing. In a similar manner, a store instruction maystore data from the data cache 706 to a location in untagged memory 716.For such a store instruction, the data may be transferred from 706 over710 to validate drop tag 714 b which outputs the untagged data. Theuntagged data output by 71 ba is then transmitted over 715 for storagein 716.

Code may be executed on the processor 702 to import the untagged datafrom 716 into the system for storage, for example, on DRAM 712 c. Thefollowing may be denote logic of the code that imports the untaggeddata:

1. the tagged data output by add tag 714 a may be stored in untrustedbuffer (tagged as public, untrusted).

2. decrypt the tagged data stored in untrusted buffer and store indecode buffer. Thus decode buffer includes decrypted data that is taggedas public, untrusted.

3. perform validation processing to ensure that the decode bufferincludes valid uncompromised data. Such validation processing may usedigital signatures as described elsewhere herein and known in the art.

4. if the decode buffer includes validate data, a second portion oftrusted code may be executed to convert the data of decode buffer taggedas public, untrusted, to data that is tagged as trusted. The trustedcode portion may include one or more instructions that, when executed,invoke rules to retag the data of decode buffer as trusted, public. Theretagged data now tagged as trusted, public may be stored in a trustedbuffer located in external DRAM 712 c.

The trusted code may include a memory instruction tagged with a specialinstruction tag giving it the authority to, when executed, invoke a rulethat retags a referenced memory location. For example, the trusted codemay include a specially tagged store instruction that stores the datatagged as public, untrusted (untrusted buffer) in a destination memorylocation (trusted buffer) with a new tag of public, trusted. Theforegoing store instruction of trust may be specially tagged, forexample, by a loader.

The following may be denote logic of the trust code that retags datafrom public, untrusted, to public trusted:

-   -   for i=1 to N        -   temp=*untrusted buffer [i];        -   trusted buffer [i]=temp;            where N is the length of the untrusted buffer and temp is a            temporary buffer used for the retagging performed. The first            instruction, temp=*untrusted buffer [i], may result in a            load instruction that loads a first element of untrusted            buffer from untagged memory 716 into temp. The second            instruction, trusted buffer [i]=temp, may be a store            instruction that stores the data tagged in temp as public,            untrusted to trusted buffer [i] with a new tag of public,            trusted. Thus, the second instruction is the instruction            that is specially tagged as noted above to have authority to            perform the data retagging from untrusted to trusted.

In a similar manner, when tagged data of 712 c is being exported orstored in untagged memory 716 (or any untagged memory source 701), codeis executed by the processor 702 that encrypts the data item andgenerates a signature where the encrypted data item and signature may besent to 714 b where the tag is removed before transmission over 715 forstoring in 716.

As a variation to the example 700, the memories 716 and 712 c may beunified and also the interconnect fabric 710 and 715 may be unified. Insuch an embodiment, the address range that the untagged memory source701 is allowed to access may be limited. For example, reference is madeto the example 720 of FIG. 58. The example 720 includes componentssimilar to those numbered as in 700 with the difference that components714 a-b, 715 and 716 are eliminated and memory 712 c includes a portionU 722 denoting a region of memory 712 c used for storing untrusted,public tagged data. The untagged memory source 701, such as an untrustedDMA and I/O subsystem, may be limited to using the bottom 16 or 256 MBof memory 722. In one embodiment, data stored in U 722 may not beexplicitly tagged but rather all data stored in U having an address inthis limited range may be implicitly tagged and treated as public anduntrusted. As a variation, an embodiment may pretag portion U 722 withpermanent tags indicating untrusted public data and the foregoingassociated permanent metadata tag cannot be modified. Rules may preventthe processor from storing other data into region U 722. Untrusted DMAoperations, for example, performed by a DMA included in 701 may belimited to writing into region U 722.

An embodiment needing to run unported I/O processing code may beexecuted on a dedicated I/O processor on the untrusted side ofcomponents. For example, reference is made to the example 730 of FIG.59. The example 730 includes components similar to those numbered as in700 with the difference of the addition of components 732, 732 a and 732b. Element 732 is the additional RISC-V processor that runs without thePUMP and metadata rule processing. Element 732 a denote the data cachefor the second processor 732 and element 732 b denotes the instructioncache for the second processor 732. The data cache 732 may be connectedto the untagged interconnect fabric 715.

As described elsewhere herein in more detail, a separate I/O PUMP may beused as another alternative to mediate between the untagged data sources(e.g., 701, 716) and tagged memory 712 c used by the processor 702.

Referring to FIG. 60, shown is another embodiment of components that maybe included in a system herein used in connection with techniques hereinto mediate between the untagged data sources (e.g., 701, 716) and taggedmemory 712 c used by the processor 702. The example 740 includescomponents similar to the example 700 with the difference thatcomponents 714 a-b are removed and replaced with intern 742 and extern744. In this embodiment, intern 742 and extern 744 may be hardwarecomponents that perform the processing described above. In particular,intern 742 may include hardware that processes received untagged dataand outputs a validated data item tagged as trusted, public. Thetrusted, public tagged data item may be communicated to over fabric 710for storage in the data cache 706 used by the processor 702 inconnection with executing instructions. Intern 742 may include hardwarethat performs validation processing of the untagged encrypted data, and,assuming successful validation, further tags the received untagged dataas trusted, public. Extern 744 may include hardware that processestagged unencrypted data and outputs a signed encrypted data item. Externmay remove the tag prior to encryption if the signed encrypted data itemis going to be used on another processor that does not perform metadatarule processing as described herein.

In a simplest case, the hardware of intern 742 and extern 744 may host asingle public-private key set where the signing and cryptography arealso performed using the single key set. The key set may be encoded inhardware used by 742 and 744. In a further variation, the hardware ofintern 742 and extern 744 may host multiple public-private key setswhere the signing and cryptography are also performed using one of themultiple key sets (each set including a different public-private keypair). The multiple key sets may be encoded in hardware used by 742 and744. Clear data included with the incoming untagged data tells theintern unit 742 which set of keys to use. Thus, intern 742 may perform alookup in a hardware data store (e.g., associative memory) including themultiple key sets to select the desired key set. Each of the multiplekey sets may be associated with a different tag so the particular keyset indicated by the clear data also indicates the particular tag thatthe tagged data will include. In this manner, the tag of the tagged dataitem output by 742 denotes that the data item is public and trusted andalso that the data item is encrypted/decrypted using a particular one ofthe multiple key sets. In the embodiment with multiple key sets, extern744 may examine the tag to determine which particular one of themultiple key sets is used in connection with encrypting and signing adata item. Thus, the intern unit 742 processing provides an isolatedhardware component that verifies received untagged data and performstagging thereby avoiding the need to have a portion of code, such as thetrust code portion noted above, with the capability to tag data.

With reference back to FIGS. 1 and 24, inputs to the PUMP 10 in stage 5include tags as described elsewhere herein. For an instruction includinga memory location as an operand of the instruction, obtaining the memoryinput and associated tag, MR tag (also sometimes referred to herein asthe Mtag), may cause an extra pipeline stall whereby PUMP 10 at stage 5cannot proceed until it has all its inputs including the MR tag. Ratherthan wait to retrieve the actual MR tag value read from memory,processing may be performed in accordance with techniques herein todetermine an expected or predicted MR tag which can then be used todetermine the R tag, the tag value for the instruction's result (e.g.,destination register or memory location, if any). In such an embodiment,a final check may be done in stage 6, the writeback or commit stage(e.g., see element 22 of FIG. 1 and commit stage as last stage 6 in FIG.24) to determine whether the predicted MR tag matches the action MR tagretrieved from memory for the operand of the instruction. The foregoingselection and use of a predicted MR tag to determine Rtag for aninstruction having a memory location as an operand may be referred to asan Rtag prediction accelerator optimization.

Referring to FIG. 61, shown is an example 800 illustrating components ofan embodiment in accordance with techniques herein for the Rtagprediction accelerator optimization. The example 800 includes the PUMP802 corresponding to the PUMP 10 at stage 5 as described elsewhereherein (e.g., FIGS. 1 and 24) with the additional features forperforming the Rtag prediction accelerator optimization. The PUMP 802includes as inputs the MR tag 804 a as well as other PUMP inputs 804 asdescribed elsewhere herein. The PUMP 802 also includes another input, aprediction selector mode 804 b, which denotes whether the PUMP 802 runsin normal processing mode (non-prediction mode where MR tag predictionprocessing is not performed) or otherwise runs in a prediction mode(where MR tag prediction processing is performed). In at least oneembodiment, the prediction mode selector 804 b may either be 0, denotingnormal processing mode for the PUMP where no predicted MR tag value isdetermined, or 1, denoting prediction mode for the PUMP where apredicted MR tag value is determined. When the prediction mode selectoris 1, the PUMP 802 may execute in prediction mode where the MR tag 804 ainput may be masked out or ignored and the PUMP 802 produces as anoutput the predicted MR tag 805 c. When the prediction mode selector is0, the PUMP 802 may execute in a normal processing mode such asdescribed elsewhere herein wherein MR tag 804 a is an input to the PUMP802 and there is no output 805 c generated.

As illustrated in the example 800, additional outputs of the PUMP 802 instage 5 include R tag 805 a and PC new tag 805 b. When using a predictedMR tag, a rule for the predicted MR tag may be determined where the rulespecifies an associated tag for R tag. When operating in predictionmode, the predicted MR tag 805 c is an additional input to stage 6 808of the pipeline. Element 808 may denote the commit or writeback stage asdescribed elsewhere herein (e.g., FIGS. 1 and 24). Thus, element 808 amay generally denote other stage 6 inputs, besides 805 a-c, as describedelsewhere herein.

In stage 6 808, additional processing 808 b may be performed when thePUMP 802 operates in prediction mode. Element 808 b indicates that acheck may be performed in stage 6 808 which compares the predicted MRtag to the action MR tag obtained from memory for the operand of theinstruction. In other words, 808 b evaluates whether the PUMP 802correctly predicted the MR tag value by determining whether thepredicted MR tag matches the MR tag obtained from memory. If thepredicted MR tag does not match MR tag as obtained from memory, then anincorrect rule was triggered and used by PUMP 802 in determining R tag805 a with the incorrect predicted MR tag. The correct rule must now beselected (in accordance with the actual MR tag) and used in determininga revised R tag. Thus, if the predicted MR tag does not match MR tag, arule cache miss is determined and cache miss handling is performed.Consistent with description elsewhere herein, cache miss handling mayinclude processing to select and evaluate the correct rule using MR tag.

Load/read and store/write instructions are examples of instructions inan embodiment that may include a memory location as an operandbenefiting from use of a predicted MR tag. Other inputs 804 to the PUMPinclude a set of other or remaining input tags besides MR tag 804 a. Forexample, one embodiment as illustrated in connection with FIG. 23 mayhave 5 input tags—PC tag, CI tag, OP1 tag, OP2 tag, and MR tag—and 2output tags—PC new and R tag. Thus the set of remaining input tags(besides MR tag) includes the following 4 tags of PC tag, CI tag, OP1tag, OP2 tag. Determining a predicted MR tag or an instruction mayinclude determining a set of one or more rules having tags values thatmatch the 4 tags (e.g., for PC tag, CI tag, OP1 tag, OP2 tag) of theinstruction. In some instances, only a single rule may include matchingtag values for the 4 input tags. In this case, the single matching rulealso specifies a value to MR tag which may be used as the predicted MRtag 805 c. Additionally, the rule may be evaluated using the 4 inputstags and the predicted MR tag to further determine the R tag 805 a.

For example, consider a memory safety policy with typical load and storeoperations. A load operation may load data from a source memory locationusing a pointer where a first rule indicates that the tag or color onthe source memory location should match the tag or color of the pointer.A store operation may store data to a target memory location using apointer where a second rule indicates that the tag or color on thetarget memory location should match the tag or color of the pointer. Fora load instruction, the first rule may be the only rule having tagvalues matching the 4 input tags for PC tag CI tag, OP1 tag and OP2 tagof the load instruction. The MR tag of the first rule may be used as thepredicted MR tag 805 c. Additionally, the R tag of the first rule may bedetermined using the set of 4 input tags and the predicted MR tag. In asimilar manner, for a store instruction, the second rule may be the onlyrule having tag values matching the 4 input tags for PC tag CI tag, OP1tag and OP2 tag of the store instruction. The MR tag of the second rulemay be used as the predicted MR tag 805 c. Additionally, the R tag ofthe second rule may be determined using the set of 4 input tags and thepredicted MR tag.

In other instances, the set of rules of the policy having tags matchingthe input tags for PC tag CI tag, OP1 tag and OP2 tag of an instructionmay include multiple matching rules with each matching rule identifyinga different allowable or candidate MR tag that may be used as thepredicted MR tag 805 c. An embodiment may use any suitable technique toselect one of the multiple allowable MR tags to use as the predicted MRtag. For example, an embodiment may select the MR tag of the set ofallowable MR tags that is the most common or likely to occur. The MR tagthat is most likely to occur may be based on previous observations orrule profiling. As an alternative, an embodiment may set the predictedMR tag to be the previous or most recent MR tag. In the worst case, ifthe predicted MR tag does not match the actual MR tag once received,cache miss handling may be performed as described herein to determinethe correct rule using the actual MR tag along with the other inputstags of the instruction.

In at least one embodiment, a class of rules for memory operations maybe created which are used when the PUMP operates in prediction mode. Theclass of rules may be referred to as a class of “predict memory tag”rules. For the “predict memory tag” rules, the MR tag 804 a is not usedas an input to the PUMP 802 and is thus not used in connection withvarious lookups performed by the PUMP. For example, the care/don't carebit vector for the “predict memory tag” rules may treat the MR tag as adon't care. Additionally, the “predict memory tag” rules may omit the MRtag as an input and rather specify predicted MR tag as an output. Asdescribed above, if there are multiple matching normal rules matching aparticular set of input tags for PC tag, CI tag, OP1 tag and OP2 tag,the single “predict memory tag” rule corresponding to the set ofmatching rules may specify a predicted MR tag as an output that is themost common or expected MR tag. In one embodiment, the single “predictmemory tag” rule corresponding to the set of matching rules may specify,as the predicted MR tag, the last or previous MR tag that was receivedby the PUMP 802.

Policy logic may decide whether to insert or use “predict memory tag”rules or not. An embodiment may keep 2 versions of each policy where afirst version includes policy “predict memory tag” rules for use whenoperating in prediction mode and a second version includes normal ornon-prediction policy rules for use when operating in normal processingmode or non-prediction mode. If the check performed in 808 b of stage 6fails for a given instruction when using a “predict memory tag” rule,the cache miss handling may perform processing to determine a matchingrule for the instruction using the normal rule set (e.g., second versionof rules described above).

In an embodiment using a RISC-V processor and architecture, theprediction mode selector 804 b may have a corresponding PUMP CSR. Use ofCSRs in an embodiment using the RISC-V architecture is describedelsewhere herein in more detail.

Referring to FIG. 62, shown is a flowchart of processing steps that maybe performed in an embodiment in accordance with techniques herein. Theflowchart 840 summarizes processing as described above in connectionwith the example 800. As noted above, the PUMP 802 illustrated in theexample 800 denotes the PUMP at stage 5 providing inputs to stage 6 ofthe processor pipeline. In at least one embodiment steps 842, 844, 846,848 and 852 of the flowchart 840 may denote processing steps performedin stage 5 as described above embodied within the PUMP and theparticular policy rules used, and steps 854, 856 and 858 may beperformed in stage 6 as described above.

At step 842, a determination is made as to whether prediction mode ison/enabled thereby denoting that the PUMP is operating in predictionmode using “predict memory tag” rules. If step 842 evaluates to no,control proceeds to step 846 where the PUMP operates in normal ornon-prediction mode using normal rules. If step 842 evaluates to yes,control proceeds to step 844 where a determination is made as to whetherthe current instruction is a memory input operation instruction. If step842 evaluates to no, control proceeds to step 846. If step 844 evaluatesto yes, control proceeds to step 848 where the PUMP operates inprediction mode using “predicted memory tag” rules. In step 848, amatching “predicted memory tag” rule for the instruction may bedetermined. In step 852, the R tag for the current instruction may bedetermined using the matching “predicted memory tag” rule from step 848.At step 854, a determination is made as to whether the predicted MR tagmatches the actual MR tag. If step 854 evaluates to no, control proceedsto step 856 to perform rule cache miss processing by invoking the rulemiss handler. If step 856 evaluates to yes, control proceeds to step 858where the R Tag, as determined with the rule including the predicted MRtag, is used as the R tag PUMP output.

As a variation from the of the example 800, reference is made to FIG. 63illustrating components of an embodiment including PUMP 802 running innormal non-prediction mode and also a second PUMP 822 that runs inprediction mode. In this example, PUMP 822 running in prediction modemay also be referred to as the MR tag prediction PUMP where theprediction mode selector 822 b is always ON (e.g., 1). Similarly, forthe PUMP 802 the prediction mode selector 804 b may also be OFF (e.g.,0). The MR tag prediction PUMP 822 may only use the “predict memory tag”rules and the PUMP 802 may only use the normal or non-prediction versionof the policy rules. In such an embodiment, the PUMPs 802 and 822 mayoperate in parallel in stage 5. Element 828 may denote the stages 5 and6 processing and components associated with the MR tag prediction PUMP822. Element 829 may denote the stages 5 and 6 processing and componentsassociated with the PUMP 802 operating in normal mode. In 829, the PUMP802 outputs are as in connection with the example 800 with thedifference that the predicted MR tag 805 c is no longer output by PUMP802. Additionally, stage 6 808 does not perform the check 808 b. Element828 may include components that perform processing in a manner similarto the example 800 with a difference being that the MR tag predictionPUMP 822 only uses “predict memory tag” rules as noted above.

Stage 6 (808) is revised to take PUMP outputs Rtag 805 a and PCnewtag805 b from MR tag prediction PUMP 822 and outputs Rtag 805 d and PCnewtag 805 e from PUMP 802. Additionally, in stage 6, selection is madebetween Rtags 805 a and 805 d and also slection is made between PCnewtags 805 c and 805 e based on whether or not the predicted MR tagmatches the actual MR tag (e.g., as denoted by 808 a). If there is amatch between the predicted MRtag and the actual MRtag (e.g., 808 aevaluates to 1 or true), the tags (e.g., Rtag 805 a and PCnew tag 805 b)from the predicted PUMP 822 are used and the tags (e.g., Rtag 805 d andPCnew tag 805 e) from the non-predicted PUMP 802 are discarded. If thereis a mismatch between the predicted MRtag and the actual MRtag (e.g.,808 a evaluates to 0 or false), the tags 805 a-c from the predicted PUMP822 are discarded and the tags 805 d-e from the non-predicted PUMP 802are used. The non-predicted PUMP 802 provides its output outputs 805 d-ea cycle later than the outputs 805 a-c of predicted PUMP 822, so whenPUMP outputs from stage 5 regarding PCnewtag and MRtag are needed asinputs to stage 6 for processing, this introduces a stall into stage 6waiting on the foregoing stage 6 inputs. The non-predicted PUMP 802 mayalso experience a PUMP rule cache miss when it is selected, in whichcase, this is handled like a typical rule cache miss as describedelsewhere within this disclosure.

Referring to the stage 6 808, elements 850 and 852 representmultiplexers. Element 808 a may denote a selector used to select aninput from each of 850 and 852 based on the logical result of whetherMRtag predicted matches MRtag. If the foregoing two tag values match,Rtag 805 a is selected as the input into 850 provided as the selectedRtag 850 a denoting the in final Rtag output of stage 6; otherwise ifthe foregoing two tag values do not match, Rtag 805 d is selected as theinput into 850 provided as the selected Rtag 850 a. Additionally, if theforegoing two tag values match, PCnew tag 805 b is selected as the inputinto 852 provided as the selected PCnew tag 852 a denoting the in finalPCnew tag output of stage 6; otherwise if the foregoing two tag valuesdo not match, PCnew tag 805 e is selected as the input into 852 providedas the selected PCnew tag 852 a.

What will now be described are techniques using coloring allocatedmemory that may be used in an embodiment in accordance with techniquesherein.

A user program, such as one coded in the C programming language, mayinclude calls to routines used in connection with memory allocation anddeallocation. For example, malloc and free are routines in the Cstandard library and may be linked into an executable of a user program.Thus, malloc and free execute as routines in the user process addressspace along with other user code that may invoke malloc and free. Mallocis invoked for dynamic memory allocation to allocate a block of memoryused by executing code. In at least one embodiment, malloc may have aninput specified on the invocation denoting the size of the memory blockto be allocated whereby malloc returns a pointer to the allocated memoryblock. A program accesses the allocated memory block using the pointerreturned by malloc. In at least one embodiment, free is invoked to freeor deallocate memory previously allocated with malloc. When a memoryblock allocated using malloc is no longer needed, the pointer (asreturned by malloc) may be passed to free as an input argument wherebyfree deallocates the memory (located as the address denoted by thepointer) so that it may be used for other purposes. User code executingon a processor in an embodiment in accordance with techniques herein mayperform such calls to malloc and free or other routines or functionssimilarly performing memory allocation and deallocation. Routines suchas malloc and free that perform dynamic memory allocation may utilizememory management metadata regarding the allocated memory. In followingparagraphs, such metadata used for memory management may be referred toas malloc metadata and is distinct and in addition to tag-based metadatadescribed herein including tags and other metadata pointed to by pointertags (e.g., where tag-based metadata that is inaccessible to executinguser code and is processed by the metadata processor or subsystem suchas described in connection with the example 1000 and elsewhere herein).Malloc metadata may include, for example, information about theallocated memory block such as the size of an allocated memory block,and a pointer to the malloc metadata portion for a subsequentlyallocated memory block.

Referring to FIG. 64, shown is an example illustrating memory allocationsuch as in connection with malloc. In the example 1100, a program mayperform a first call to malloc to allocate a first block of memory of arequested size. In response, malloc may allocate memory block 1102 b ofthe requested size and return pointer P1 denoting the starting addressfor the memory block 1102 b. The user program may then store data to,and read data from the allocated memory block 1102 b using the pointerP1 or another address based on an offset from P1. Additionally, forpurposes of dynamic memory management, malloc may also allocate storage1102 a for its own malloc metadata for each memory block allocated.Element 1102 a denotes the memory portion allocated and used by mallocfor storing the malloc metadata for the allocated memory block 1102 b.In a similar manner, the user program may subsequently perform a secondcall to malloc to allocate a second memory block. Element 1104 a denotesthe memory portion allocated by malloc responsive to this second callwhere 1104 a is used for storing malloc metadata. Element 1104 b denotesthe second memory block allocated where P2 is the pointer returned tothe user program to access the second memory block. In a similar manner,the user program may subsequently perform a third call to malloc toallocate a third memory block. Element 1106 a denotes the memory portionallocated by malloc responsive to this third call where 1106 a is usedfor storing malloc metadata. Element 1106 b denotes the third memoryblock allocated where P2 is the pointer returned to the user program toaccess the third memory block.

After an allocated block of memory, such as 1102 b, is no longer neededby the executing code, the code may perform a call to free to free thememory block 1102 b so that such memory block 1102 b is deallocated andmay be used for other purposes. Pointer P1 may be returned when makingsuch a call to free. In a similar manner, when memory blocks 1104 b-care no longer needed, calls to free may be made specifying,respectively, pointers P2 and P3.

Through a pointer such as P1 returned by malloc to executing user code,the user code may inadvertently or intentionally access the mallocmetadata since the address of the memory portion 1102 a holding themalloc metadata is mapped into the executing code's address space. Forexample, the user code may assign another pointer P4 an address inmemory portion 1102 a (e.g., P4=P1-2) and then read or write to thememory location identified by the pointer P4. Thus, the user code may,for example, overwrite the malloc metadata stored in 1102 a and readmalloc metadata stored in 1102 a. In this manner, performing a write tothe memory location at the address identified by P4 may corrupt themalloc metadata portion 1102 a. More generally, the foregoing may beperformed by user code in connection with any of the malloc metadataportions 1102 a, 1104 a and 1106 a.

In connection with a call to free, user code may specify a pointer thatdoes correspond to the starting address of an allocated memory blockpreviously allocated using malloc. For example, the user code mayperform a call to free specifying the foregoing pointer P4 as anargument rather than P1, P2 or P3. Assume, for example, malloc allocatesan X byte block (e.g., X being a non-zero integer) for each mallocmetadata portion 1102 a-c in connection with a call to malloc. Theroutine free may perform processing under the assumption that memorylocations from the first address (P4-X) to the second address (P4-1)denote, respectively, starting and ending address spanning a mallocmetadata portion such as 1102 a. In this case, processing performed byfree may be using a corrupted malloc metadata portion 1102 a resultingin, for example, unexpected runtime performance and/or dynamic memorymanagement errors.

An embodiment may use techniques described herein to protect the mallocmetadata portions 1102 a, 1104 a and 1106 a to avoid corruption throughoverwrites performed by other executing code, such as the user code.Such techniques may include tagging code and/or data with particularcolors or tags and enforcing rules to allow only desired access andoperations such as described elsewhere herein.

With reference to FIG. 65, in at least one embodiment, memory portionsused by malloc and free may be colored or tagged with a first tag usedby metadata processing as described herein and other memory portionsused by user code (as allocated by malloc) may be colored or tagged witha second different tag used by metadata processing as described herein.In the example 1100, data portions used by malloc and free (containingmalloc metadata) may be colored or tagged red, and user data portions(memory blocks allocated by malloc for use by user code) may be coloredor tagged blue. An embodiment may have at least one tag or colorreserved exclusively for use in coloring or tagging memory locationsused by malloc and free. In this example, red is the reserved color usedfor tagging memory locations used by malloc and free. As describedelsewhere herein, an embodiment may also reserve one or more colors ortags for executing user code. In at least one embodiment, all memoryallocated for use by a user program may be tagged with the same color.As a variation, an embodiment may use a different tag for each call tomalloc and thus a different color for each separate memory blockallocated. In this example 1110, for simplicity of illustration, only asingle color blue is used to tag all memory blocks allocated by mallocfor a user program.

Element 1111 may denote tags specified for corresponding memorylocations 1113. Elements 1112 a, 1114 a and 1116 a respectively denotetags for malloc metadata portions 1102 a, 1104 a and 1106 a. Elements1112 b, 1114 b and 1116 b respectively denote tags for memory blocks1102 b, 1104 b and 1106 b allocated by malloc for user by user code viacalls made to malloc as described above.

Elements 1112 a, 1114 a and 1116 a denote that each memory location,respectively in 1102 a, 1104 a and 1106 a is tagged as red. Elements1112 b, 1114 b and 1116 b denote that each memory location, respectivelyin 1102 b, 1104 b and 1106 b is tagged as blue.

Generally, an embodiment may use instruction tagging, colored pointers,or a combination of the foregoing, in connection with triggering rulesthat color the memory blocks of 1113 with tags denoted by 1111, and alsoenforce a memory safety policy whereby only malloc and free are able toaccess malloc metadata areas 1102 a, 1104 a and 1106 a and user codecannot.

In a first embodiment, code of malloc and free may be tagged (e.g.,instruction tagging) such as by a loader, with a special instruction tag(e.g., CI tag). Both malloc and free may be tagged with the same uniqueor special instruction tag (e.g., malloc and free code tagged with thesame CI tag of tmem) or may each be tagged with their own unique orspecial instruction tag (e.g., malloc code tagged with tmalloc and freecode tagged with tfree). Code of malloc may include store instructionsthat, when executed, trigger rules that perform coloring such as in theexample 1110. Code of free may include store instructions that, whenexecuted, trigger rules that reinitialize or deallocate a mallocmetadata portion (e.g., 1102 a, 1104 a and 1106 a) or a previouslymalloced memory block (e.g., 1102 b, 1104 b and 1106 b) such as byretagging each memory cell of the block or malloc metadata portion withan F tag representing free memory. Also, in the first embodiment, thememory safety policy may include rules triggered by execution ofparticular instructions, such as the load and store instructions,whereby the rules only allow instructions tagged with the specialinstruction tag(s) noted above to 1) access malloc metadata portions1102 a, 1104 a and 1106 a and 2) perform the memory block coloring as inthe example 1110. Such rules may generally check the CI tag to ensurethat each instruction coloring or accessing a memory cell in any of 1102a, 1104 a and 1106 a has the special instruction tag denoting malloc orfree.

In a second embodiment, rather than use special instruction tags, anembodiment may use colored pointers with rules of the memory safetypolicy triggered by execution of particular instructions, such as theload and store instructions. The loader may tag pointers of malloc andfree that reference malloc metadata portions 1102 a, 1104 a and 1106 awith the color red. Code of malloc may include store instructions that,when executed, trigger rules that perform coloring such as in theexample 1110. Code of free may include store instructions that, whenexecuted, trigger rules that reinitialize or deallocate a mallocmetadata portion (e.g., 1102 a, 1104 a and 1106 a) or a previouslymalloced memory block (e.g., 1102 b, 1104 b and 1106 b) such as byretagging a memory cell with a tag F representing free memory. Thememory safety policy may include rules triggered by execution ofparticular instructions, such as the load and store instructions,whereby the rules only allow access to malloc metadata portions 1102 a,1104 a and 1106 a with instructions referencing a memory cell using ared colored pointer. Such rules may generally check the MR tag to ensurethat the memory instruction accessing a memory cell in any of 1102 a,1104 a and 1106 a uses a pointer with a first color that matches asecond color of the memory cell.

In a third embodiment, both special instruction tags and coloredpointers as described above may be utilized in combination. Following isan example of instructions and rules that may be used in such a thirdembodiment. Consistent with other discussion herein, following examplesuse rules based on 5 input tags to metadata processing for PC (programcounter), CI (current instruction), OP1 (operand 1 of the currentinstruction), OP2, operand 2 of the current instruction), MR (memorylocation, if any, referenced in the current instruction), and twopropagated or generated tags for PCnew (new PC tag for next PC for nextinstruction) and R (tag for result of current instruction; used to tagdestination register or memory location into which the result of thecurrent instruction is stored). Additionally, “-” denotes a don't carefor a tag. In such an embodiment, the loader may tag instructions ofmalloc with the special tag tmalloc and may tag instruction of free withthe special tag tfree. Colored pointers may be created using triggeredrules noted below.

In connection with malloc, metadata rule processing triggered byexecuting the code portion of malloc may generate a tag for the pointerto a newly allocated memory block such as 1102 b via a first ruleinvoked as a result of a store instruction in the code portion ofmalloc. For example, malloc C code may be “P1=next free” where next freeis a pointer to the next free memory location in 1113 and the storeinstruction may be “move R1, R2”, where register R1 is the sourceregister that contains the address next free and register R2 is thedestination register that is the pointer P1. Register R1 may be OP1(having OP1 tag) and register R2 may be the result or destinationregister (having R tag propagated or generated as a result of a firedrule). The code portion of malloc may include instructions, such as theforegoing move instruction, also tagged with a special tag, tmalloc,denoting the instruction is included in malloc code. In at least oneembodiment, the loader may tag instructions of malloc with the specialcode tag, tmalloc. The first rule may tag pointer P1 to the allocatedmemory block 1102 b with the tag blue. The first rule triggered as aresult of the above move instruction in malloc may be:

-   -   my rule1A: (-, t malloc, blue-predecessor, -)→(-, blue)        The above rule only fires when the CI tag is tmalloc and thus        for tagged move instruction in malloc. Assuming that the pointer        used by malloc is P1, the above my rule 1A tags P1 stored in        register R2 with the tag or color blue to denote it is a pointer        to a blue memory location (e.g., memory location tagged with        blue tag).

The pointer P1 tagged with blue may then be used with another secondstore instruction of malloc to write a 0 or some other initial value toeach word in the allocated memory block 1102 b. For example “*P1=0” maybe included in malloc C code resulting in “Store R3, (R2)” where R3 is asource register operand OP1 containing zero (0), and R2 is the OP2register that contains the address P1. In this store instruction, “(R2)”is operand MR and also denotes a memory location that is the target ordestination of the store instruction. Additionally, the above storeinstruction in malloc may also be tagged tmalloc and may result intriggering a second special store rule as follows:

-   -   store 2A: (-, t malloc, -, blue, F)→(-, blue)        prior to returning the tagged pointer P1 to user code that        invoked malloc.

The above store rule 2A only fires when CI tag is tmalloc, the pointeror address in R2 (denoting P1) is tagged as blue, and when the memorylocation MR pointed to by P1 has an F tag. The foregoing memory location*P1 is assumed tagged with “F” prior to coloring the memory locationblue. In this example, F denotes a free memory location. The resultingMR tag for the memory location denotes a blue tag for the memorylocation.

Thus, malloc may include code that results in triggering the above-notedsecond rule for each memory location of a memory block being allocated.

Malloc may also include code that triggers additional rules describedbelow (e.g. that are similar to move (my) rule 1A and store rule 2Aabove) for use in initializing malloc metadata portions 1102 a, 1104 a,and 1106 a. For example, malloc C code may be “(P1-2)=MD area” where MDarea is a pointer into the malloc metadata area 1102 a and the moveinstruction may be “move R7, R8”, where register R7 is the sourceregister that contains the address “P1-2” and register R8 is thedestination register that is the pointer MD area. The rule triggered bythe above move instruction may be:

-   -   my rule 1B: (-, t malloc_md, -)→(-, red)        to tag the MD area pointer red.

Malloc may also include code that triggers store rule 2B noted below(that is similar to the store 2A rule above) to tag each memory locationof a malloc metadata portion such as to store tags 1112 a, 1114 a, 1116a, respectively for malloc metadata portions 1102 a, 1104 a and 1106 a.For example assume size is an integer denoting the size of the mallocedmemory block 1102 b and that “*(P1-2)=size” is included in malloc C coderesulting in “Store R6, (R7)” where R6 is a source register operand OP1containing the size value), and R7 is the OP2 register that contains theaddress P1. In this store instruction, “(R7-2)” is operand MR and alsodenotes a memory location in MR 1102 a that is the target or destinationof the store instruction. The store rule 2B may be:

-   -   store 2B: (-, t malloc md, -, red, F)→(-, red)    -   store 2D: (-, t malloc md, -, red, red)→(-, red)        which performs the store if the store instruction is tagged        tmalloc, if the R7 register containing address P1 is tagged red,        and if the MR operand is tagged as F. It should be noted that an        embodiment may also include store rule 2D (store 2D) noted above        that is a variation of store 2B rule noted above whereby store        2D rule may be used in cases where updating the metadata value        is desired.

At a later point in time, free may include code, such as “*P=0” thatresults in triggering store rule 3 noted below to retag a memorylocation of a previously malloced memory block (e.g., memory blockallocated for user code use), such as when freeing or deallocating bluecolored block 1102 b. The loader may color or tag instructions of freewith tfree. The routine free may include the C code statement “*P=0”that results in “Store R4, (R1)” where R4 is source register operand OP1containing zero, R1 is the OP2 register containing the address of thememory location to be initialized, and “(R1)” denotes a memory operandMR with R1 containing the address to the memory location. The store rule3 may be:

-   -   store rule 3: (-, t free, -, blue, blue)→(-, F)

Thus, free may include code that results in triggering the above-notedthird rule for each memory location of a memory block being deallocated,where the memory block was previously allocated using malloc for use byuser code (e.g., used for storing data other than malloc metadata).Store rule 3 checks to ensure that the CI tag=t free and that both thememory location and pointer thereto have the same color, blue.

It should be noted that the MR tag of “blue” may generally be any colorpreviously used by malloc to color an allocated user memory block.

Code of free may also include code that triggers move (my) rule 1C andstore rule 4 described below in connection with retagging each memorylocation of a malloc metadata portion such as 1112 a. Code of free mayinclude code that triggers move (my) rule 1C noted below which issimilar to move (my) rule 1B above. Move (my) rule 1C may be:

-   -   my rule1C: (-, t free, -, -, -)→(-, red)        to tag a red pointer for use by free in connection with        retagging using store rule 4.

Store rule 4 below (that is similar to the store rule 3 above) may betriggered to retag each memory location of a malloc metadata portionsuch as retag 1112 a, 1114 a, 1116 a, respectively for metadata portions1102 a, 1104 a and 1106 a. The store rule 4 may be:

-   -   store rule 4: (-, t free, -, red, red)→(-, F)        which performs the store if the store instruction is tagged        tfree, and if the MR operand uses a pointer tagged as red. The        memory location is tagged with “F” to now denote it as free.

In a fourth embodiment, PC tagging may be used to provide malloc andfree to provide malloc and free with sufficient privilege, access orauthority to read data from and write data to malloc metadata portions1102 a, 1104 a and 1106 a and also exclude other code from accessing theforegoing metadata portions. PC tagging is described elsewhere herein,for example, in connection with the example 430 with providing differentprivilege, access or authority on a per process basis using different PCtag values. In a similar manner, a special or unique PC tag value may beused to provide malloc and free with authority to perform load and storeoperations with respect to malloc metadata portions 1102 a, 1104 a and1106 a. To further illustrate, malloc may include instructions taggedwith tmalloc (e.g. CI tag=tmalloc when instruction is executed). Mallocmay also include code that, when executed, triggers application of arule that propagates or produce a particular PC tag as an outputdenoting a privilege or authority to access malloc metadata portions1102 a, 1104 a and 1106 a. Malloc may include a first instruction INS1such as:

-   -   Add 0, R2        where R2 is an address in the malloc metadata portion, such as        address P6 in area 1102 a, and (R2) denotes the memory location        having address P6 in 1102 a that is colored red. The foregoing        instruction INS1 when executed, may result in generating PCnew        having a tag value such as X1 where X1 denotes the privilege        needed to access 1102 a. In this case, the rule triggered for        the above first instruction INS1 may be:    -   add: (-, tmalloc, --, red, -)→(X1, red)        to color R2 with the color red and also set the PC to X1 to        denote read/write access to the memory location having the        address stored in R2 (e.g., address P6). Subsequently, malloc        may include a second instruction, INS2, of “store R3, (R2)” to        store a value from register R3 (e.g., OP1) into the memory        location having address P6 (P6 is stored in R2). The rule        triggered for the above second instruction INS2 may be:    -   store: (X1, tmalloc,-, red, red)→(PCdefault, red)        where PCnew is cleared or reset to be PCdefault, a default PC        tag that does not denote privilege to access malloc metadata        portion 1102 a. Thus, in this particular example, the first ADD        instruction triggers a rule to grant malloc the privilege or        authority for read/write access to 1102 a. After the above        second instruction of malloc is executed that performs a write,        the PC tag propagated removes the privilege or authority from        malloc for read/write access to 1102 a. As a variation, an        embodiment may include a version of malloc with a prolog        including an instruction that triggers a rule to grant malloc        read/write access to 1102 a by generating a PCnew tag of X1        (e.g, prolog includes Add instruction INS1 that triggers the        rule noted above). At the end of malloc before returning, an        epoligue may be executed that includes an instruction that, when        executed, triggers a rule to remove malloc's read/write access        to 1102 a by generating a PCnew tag of PCdefault (e.g., epilogue        includes store instruction INS2 that triggers the rule noted        above).

In a similar manner, free may include instructions that invoke rules togenerate or propagate a PCnew tag value to provide free with access to1102 a. The rules applied may propagate or produce a particular PC tagas an output that denotes a desired access, privilege or authority basedon the particular process whereby the particular allowed privilege,access or authority may be represented by different PC tag values.

It should be noted that the foregoing illustrates a single color of bluefor all malloced memory blocks and a single color of red for all mallocmetadata portions. More generally, as described elsewhere herein, mallocmay be provided with the authority to generate an unbounded number ofnew colors as may be needed for coloring different portions of heapmemory. As discussed elsewhere herein, for example, malloc may be givenand initial predetermined set of one or more colors or tags and maygenerate subsequently needed tags from the initial predetermined set.For example, malloc's initial predetermined set may include yellow or Yand red or R. For an executing process, malloc may generate a freshY-based tag (e.g., Y1, Y2, Y3, . . . ) for each call to malloc toallocate a new memory block used by user code (e.g., other than formalloc metadata storage). Thus, a different Y-based tag may be used tocolor each malloced memory block 1102 b, 1104 b and 1106 b (e.g., 1102 bcolored with Y1, 1104b colored with Y2, 1106b colored with Y3). Mallocmay generate a fresh R-based tag (e.g., R1, R2, R3, . . . ) for eachdifferent malloc metadata portion created for each call to malloc. Thus,R-based tags may be used to color malloc metadata portions 1102 a, 1104a, 1106 a each with a different R-based tag (e.g., 1102 a colored withR1, 1104 a colored with R2, 1106 a colored with R3). The current or lastR-based tag and the current or last Y-based tag used by malloc may bestored as state information via rules triggered when executing mallocinstructions. For example, malloc may include an instruction whichtriggers a rule that stores the last Y-based tag, Y9, as the tag of afirst memory location. Y9 may be generated as the Rtag. A subsequentinstruction may again reference the same first memory location taggedwith the saved last Y-based tag, Y9, where the subsequent instructiontriggers a rule that 1) generates a new tag, Y10, based on the lastY-based tag, Y9, and 2) saves the tag Y10 as the tag on the first memorylocation. Y10 may be generated as the Rtag. The rule triggered by thesubsequent instruction may indicate to determine Rtag, for example, asMRtag+1, where MRtag is Y9 for the subsequent instruction.

What will now be described are techniques that may be used as anoptimization in connection with metadata processing using hardwareaccelerated miss handling. Generally, some policies used in embodimentherein may cause frequent rule cache misses and the cache miss handlersfor such policies may take many cycles to run. In some policies, therelationships between various rule inputs may be rather simple in termsof logically determining a result or outcome and may therefore behardwired and resolved quickly with dedicated hardware.

As a result, such policies implemented using hardware (HW) rule cachemiss handler may be resolved in a much shorter amount of time thanothers not using such hardware acceleration. In such an embodiment,policy components, such as the cache miss handler for one or moreselected policies, may be implemented with dedicated hardware. Thus, anembodiment in accordance with techniques herein may use suchhardware-supported policies alone, or in combination with,software-defined policy components using a software rule cache misshandler.

As one example, consider a memory safety policy that uses memory safetycoloring. In connection with a memory safety policy such as describedelsewhere herein, memory cells and pointers may be colored whereby rulesinvoked in connection with both load and store operations may only allowa memory reference where a pointer color matches that of the memorycell. For example, rules triggered for a load instruction may be used toenforce a policy where the pointer color (e.g., of a register tag wherethe register is an operand such as OP1) is equal to a memory-cell color(e.g., memory location tag such as Mtag). The memory safety policy maychallenge capacity by filling the PUMP rule cache with many differentconcrete rules that simply capture this equal color relationship formany colors, increasing the capacity miss rate. In some embodiments asdescribed herein without preloading the rule cache, compulsory rulecache misses are required to insert every one of these rules. Sincememory safety policy rules may be commonly triggered in connection withexecuting user code, memory safety policy rules may be supported usingHW rule cache miss handler rather than a software rule cache misshandler.

In such an embodiment, the HW rule cache miss handler may generate orcalculate a new rule inserted into cache upon the occurrence of a rulecache miss. For example, the miss handler for memory safety may beimplemented using hardware as a HW rule cache miss handler that, forload instructions, compares OP1tag to the Mtag If OP1tag equals Mtag,the HW rule cache miss handler may generate a new rule with Rtagassigned Mtag. For example, if the pointer PTR is red and the memorycell pointed to by PTR is red, the instruction invoking the rule isallowed and the resulting tag Rtag should be red. To generate theforegoing as a new rule to be inserted in the rule cache, the HW cachemiss handler may first compare OP1tag to Mtag. If they are not equal,there has been a rule violation and the instruction is not allowed(e.g., cause processor to stop execution). If the HW rule cache misshandler determines OP1tag is equal to Mtag, the HW rule cache misshandler may generate as outputs of hardware the new rule includingopcode=load, OP1tag=red, Mtag=red, and Rtag=red (all other tag inputsand outputs of the rule may be don't cares), where the generated rulemay then be inserted into the rule cache.

Referring to FIG. 66, shown is an example illustrating a hardwareimplemented cache miss handler in an embodiment in accordance withtechniques herein. The example 1300 includes 1301 which illustratesinputs 1302 a input to the PUMP rule cache 1302 (e.g., FIG. 22) toperform a lookup to determine whether a rule matching the inputs 1302 ais in the cache. If so, the outputs 1302 b are determined based on therule stored in the cache. Consistent with discussion elsewhere herein,the inputs 1302 a may include the opcode and input tags—PCtag, Cltag,OP1tag, OP2tag, Mtag. The outputs 1302 b may include output tags of therule such as PCnew tag and Rtag. In connection with an embodiment inaccordance with techniques herein implementing the cache miss handler insoftware, upon the occurrence of a rule cache miss, the software cachemiss handler may be invoked whereby code of the miss handler executesand calculates a new rule for the inputs 1302 a causing the current rulecache miss. The cache miss handler first determines whether the inputscoincide with an allowable rule (e.g., for memory safety load rule, doesOPltag equal Mtag), and if so, calculate the outputs for the particularinputs 1302 a (e.g., determine the Rtag as Mtag) thereby generating arule for the inputs 1302 a. The new rule (which is based on thecombination of the inputs 1302 a and the calculated outputs of the misshandler) is inserted into the rule cache. Consistent with discussionelsewhere herein, the new rule may include the opcode, input tags—PCtag,CItag, OP1tag, OP2tag, Mtag—and output tags—PCnewtag, Rtag.

Element 1303 illustrates the HW rule cache miss handler 1304 that may beused in an embodiment in accordance with techniques herein rather thanthe software rule cache miss handler. In such an embodiment, the HW rulecache miss handler 1304 may be implemented using dedicated hardwareincluding, for example, gate-level logic and other hardware components.In such an embodiment, the HW miss handler 1304 may take the same inputs1302 a as the PUMP rule cache 1302 and may generate, using its hardware,the same outputs 1302 b that would be output the PUMP rule cache.Subsequently, a new rule may be formed by combining the opcode, inputtags and output tags as noted above. The new rule may then be stored inthe PUMP rule cache (e.g. FIG. 22).

In at least one embodiment, the HW rule cache miss handler for thememory safety policy may be implemented as described above in hardware(e.g., using gate level logic) that, to load a rule into cache, maysimply copy the memory-cell tag from the Mtag to the Rtag andimmediately perform the PUMP rule insertion. Note in this simple case,there is no need to dereference memory and perform any data structureoperations in memory.

Additionally, in at least one embodiment, memory safety may implementthe memory cell's tag as a pair of tags: (1) memory-cell color tag, (2)pointer-color tag on pointer in the memory cell. Memory-safetyacceleration may include dedicated caches to perform the combination ofthe Mtag and OP2tag into the new Rtag on a store, and to perform theextraction of the pointer-tag from an Mtag pair to place onto the Rtagfor a load. Misses to these caches may use simpler, dedicated softwarehandlers. While the foregoing is described for a single (non-composite)policy such as memory safety, the same general technique may be appliedto a component of a composite policy on the UCP.

An embodiment may also perform hardware acceleration using HW rule cachemiss handler for a limited common subset of rules such as those that areexpected to be commonly referenced. For example, in memory safety, therules for load/store and propagation during arithmetic are the moststandard and stylized. Other, uncommon rules exist for initiallycoloring memory regions and reclaiming memory regions on free. Suchuncommon rules may result in using the typical rule miss handler asdescribed herein rather than being implemented with hardware support.

In at least one embodiment, a HW rule cache miss handler may directlyimplement a mapping function as gate-level logic. For example, such gatelevel logic may map an input tag to an output tag for rule such asmapping Mtag to Rtag for store instruction rules of the memory safetypolicy. As another example, a HW rule cache miss handler for a CFI(control flow integrity) policy may use gate level logic to make the tagof the control flow target or destination be a pointer to the set ofallowed callers (e.g., source locations or addresses allowed to transfercontrol to the tagged particular control flow target or destination),allowing the CFI HW rule cache miss handler to read through the set fora match. As yet another example, a stack protection policy may encodethe stack-frame-code tag and the associated stack-frame-memory-cell tagin a manner that allows hardware derived one from the other (e.g. theycould differ by only a few bits, and this could be arranged even if thetags were pointers by allocating the stack-frame-code tag pointer andthe stack-frame-memory-cell tag pointer together); consequently, a HWrule cache miss handler enforcing a stack protection policy would beable to determine the tag to create (in case of creating a tag from thestack pointer), or demand on the memory reference (in the case of a reador write) within such code.

As a variation from using HW rule cache miss handlers to calculate ordetermine a new rule which is then inserted into the PUMP rule cache, anembodiment may actually hardwire the logic of one or more rules of apolicy where such rules are completely embodied and enforced in hardwareand therefore not stored in the PUMP rule cache. For example, rather usea HW rule cache miss handler and PUMP rule cache for a policy, anembodiment may use hardware to enforce and encode the rules of thepolicy (e.g., rules of policy embodied in hardware such as gate levellogic and circuits). In such an embodiment using both the PUMP rulecache and HW specified rules, a rule lookup may be performed of both thePUMP rule cache and also the HW specified rules. In this case, a misshandler (e.g., either a HW rule cache miss handler or a software misshandler) may be invoked to determine/calculate a new rule responsive tonot finding a rule for the particular inputs in either the PUMP rulecache or the HW specified rules.

Composite policies present additional challenges and opportunities.Consistent with discussion elsewhere herein, a composite policy includesmultiple policies simultaneously enforced for an instruction. Thechallenge is that the composite policies need resolution of severaldifferent policy components. The opportunity is that the entire sequenceof resolution for the composite policy may be hardware supported usingHW rule cache miss handlers for all the different policy components ofthe composite policy with the data cache, UCP caches (per UCP cache perpolicy component in the composite policy), and CTAG cache. From priorexperience, a common challenge is where newly allocated memory (e.g.,using malloc), hence new memory color tags, causes compulsory rule cachemisses. In these cases, the memory safety policy component needs newrules, but the other components may likely already have their rules inthe UCP caches. With hardware acceleration through HW rule cache misshandlers for the top-level composite policy and for the memory-safetycolor matching, memory rule resolution may be performed with a smallfinite state machine running in hardware and consulting the caches(e.g., data cache, UCP caches and CTAG cache) rather than requiringhundreds-to-thousands of cycles to resolve the rules runningsoftware-based miss handler code.

In at least one embodiment, the UCP caches may be decomposed bycomponent policy and all resolved in parallel to produce the compositeset of tag results to then be fed back into the CTAG cache. If all thepolicies may be resolved either by their UCP cache or by a simplehardware rule like the one for memory safety, the total time for lookupfor the UCP caches would be that of a single policy rather than beingproportional to the number of policies. This works perfectly if thenumber of component policies is fixed and matched to the hardwareprovided. Nonetheless, a slight variant simply distributes the componentpolicies across the fixed, available number of UCP caches, so that thenumber of sequential UCP cache resolutions is only the ratio of thenumber of component tags to physical UCP caches.

Referring to FIG. 67, shown is an example 1310 illustrating use of HWrule cache miss handlers in connection with a composite policy that maybe used in an embodiment in accordance with techniques herein. In thisparticular example, 3 policies comprise the composite policy whereby all3 policies are simultaneously enforced for the same instruction althoughmore generally a composite policy may include any number of policies andis not limited to 3. Elements 1314 a-c are HW rule cache miss handlersfor the 3 policies comprising the composite policy. Inputs 1312 may beprovided to each of the HW rule cache miss handlers 1314 a-c which,respectively, determine or calculate rule outputs 1316 a-c for theparticular policy (e.g., HW rule cache miss handler 1314 a determinesoutputs 1316 a including Rtag and PCnew tag for policy A; HW rule cachemiss handler 1314 b determines outputs 1316 b including Rtag and PCnewtag for policy B). Subsequently, the outputs 1316 a-c may be combinedinto a single composite result 1318 denoting a composite Rtag and PCnewtag for the 3 policies. The combining of the outputs 1316 a-c todetermine composite results 1318 may also be implemented using hardwareor software. A new rule may be inserted into the cache where the newrule includes the inputs 1312 (e.g., opcode and inputs tags) for theparticular instruction triggering the rule cache miss handling alongwith the composite results 1318 (e.g., composite values for Rtag andPCnew tag).

Additionally, although not illustrated in the example 1310, anembodiment may use the UCP caches and CTAG cache in combination with theHW rule cache miss handlers 1314 a-c in an embodiment in accordance withtechniques herein. As described elsewhere herein (e.g., in connectionwith FIGS. 21, 23 and 24), each of the policies A, B and C may have itsown UCP cache caching results of the most recent policy result tags(e.g., UCP cache for policy A stores recently calculated result tags bymiss handler 1314 a—PCnewtag and Rtag results—based on the combinationof opcode and input tags of an instruction). As described elsewhereherein (e.g., in connection with FIGS. 21, 23 and 24), the CTAG cachemay store composite results for Rtag for a particular combination ofindividual Rtag values as may be output from multiple composite policiessuch as policies A, B and C. The CTAG cache may also store compositeresults for PCnewtag for a particular combination of individual PCnewtag values as may be output from multiple composite policies such aspolicies A, B and C. Thus, the hardware that generates the compositeresults 1318 from outputs 1316 a-c may use information from the CTAGcache to determine the composite results 1318. Additionally, the HW rulecache miss handlers 1314 a-c may also have as inputs information fromthe UCP caches for the policies A, B and C.

As an alternative to having HW rule cache miss handlers for all 3policies of the composite policy as in the example 1310, an embodimentmay selectively choose to implement HW rule cache miss handlers for oneor more, but less than all, such policies comprising the compositepolicy. In such an embodiment, a portion of the rule cache miss handlersmay be implemented in hardware and the remaining portion of the rulecache miss handlers of the composite policy may be implemented insoftware as described elsewhere herein.

It should be noted that some policies as described herein may allocatenew tags such as, for example, in connection with a memory safetypolicy. In at least one embodiment, the HW rule cache miss handler for apolicy such as memory safety that may allocate new tags may be providedwith a FIFO-based cache of new tag values that the HW-based handler mayuse (e.g., cache of tags that may be used as the newly allocated tagvalues generated. If the tag allocated is a pointer denoting an address,then the cache includes addresses or pointers rather than the tagvalues). In this manner, the HW rule cache miss handler may performallocation simply by reading the top entry from the FIFO-based cache.Periodically, software handlers may be executed the metadata processingdomain to refill the FIFO-based cache with new tags available forallocation.

Embodiments are described herein where there is complete and strictisolation between the metadata processing domain and the “normal” codeprocessing of the user code or execution domain. As a variation, anembodiment may take a more relaxed approach and expand the foregoingstrict isolation model which still does not allow modification orwriting of information by the user code or execution domain to themetadata processing domain but may allow information/values to bereturned by the metadata domain to the user code or execution domain.

What will now be described are techniques that may be included in atleast one embodiment which may utilize the foregoing more relaxedapproach whereby the metadata processing domain returns a value that maybe used or referenced by code executing in the normal code processing orexecution domain (e.g., metadata processing returns a value that is aninput to the normal or user code execution domain). For example, asdescribed elsewhere herein, an embodiment may use malloc and freeroutines where such routines have their code tagged with instructiontags providing them with unique capabilities needed so that code ofmalloc and free, when executed, trigger rules which allow malloc andfree the ability to perform their processing accessing malloc metadata,generating new color tags, tagging user data areas with such new colortags, and the like. The foregoing provides such privileges or abilitiesuniquely assigned to malloc and free at the exclusion of other code,such as user code. Now consider such an embodiment where malloc and freeperform their processing and code tagging is utilized to specially tagmalloc and free code by the loader with a special code tag(s) uniquelyidentifying such code as belonging to malloc and free with specialexecution privileges. In such an embodiment, it may be the case thatuser code making a call to free, for example provides a pointer PTR1which has been corrupted or otherwise does not point to the beginning ofthe previously allocated storage area which is now being deallocated.PTR1 may be presumed, by free, to point to the first location of theuser data area previously allocated by malloc. Free may assume apredetermined structure to the user data area and associated memorylocations of the memory heap such as described in connection with FIGS.64 and 65, for example, where the malloc metadata is stored in apredetermined location relative to the allocated user data area.

What will now be described are techniques that may be used in anembodiment to have the PUMP return a value to the code execution domain.

With reference to the example 1200 of FIG. 68 shown are elements 1111and 1113 as described in connection with example 1110 of FIG. 65 furtherannotated with pointers PTR1 and PTR2 discussed below. Assume that usercode invokes free with PTR1 with the intent to deallocate memory block1102 b. P1 may denote the pointer or address expected by free. However,PTR1 in this example may denote a corrupted or incorrect address thatgenerally denotes a different address other than P1 (e.g., PTR1 mayidentify a location in memory 1102 b, or may denote an address not evenpointing into the heap). Although PTR1 has been corrupted or otherwisedoes not point to the correct memory location P1, free may performprocessing using PTR1 to access malloc metadata using relativeaddressing relative to PTR1 where the malloc metadata is assumed toexist in its predefined structure, format or layout. For example, themalloc metadata area used by free may be presumed to be locatedimmediately prior to the allocated user data portion as in FIGS. 64 and65. In such a case, the code of free may determine the malloc metadatait uses in processing to deallocate a particular memory block is locatedat a particular offset OFF1 prior to PTR1 based on a predeterminedlayout. For example, with reference to FIG. 68, free may presume thatPTR1=P1 where PTR1 may be provided by the user code on the call to free.Free may use relative addressing as described above based on apredefined data layout that the corresponding malloc metadata 1102 a forthe memory block 1102 b to be deallocated should begin at the memorylocation with address PTR2=PTR1−OFF1. In this example, PTR1 does notequal P1 and PTR1 actually points to somewhere in the allocated memoryblock 1102 b so that the address calculation PTR2=PTR1-OFF1 is also inthe user allocated memory block 1102 b (PTR2 denotes the expectedbeginning of the associated malloc metadata used by free).

In such a case where PTR1 provided by the user code on the freeinvocation does not point to the expected location P1 and whereby PTR2denotes the presumed beginning of malloc metadata used by free, code offree may incorrectly access data stored in memory block 1102 b usingsuch data as its malloc metadata which causes a violation, interrupt ortrap (e.g., may be due to a rule violation detected by the PUMP, orother code execution error condition during execution of free). Thus,the execution of code executing in the user process space or domain maybe aborted due to the foregoing violation during execution of theroutine free as invoked in the call to free using PTR1 from user code.Rather than have routine free cause the foregoing abort of user code, itmay be desirable to allow code of free to query the PUMP, or moregenerally, the metadata processing to return a value. The returned valuemay be, for example, a Boolean denoting whether the color associatedwith PTR2 (as will be used by code of free to access malloc metadata)actually points to a valid or expected malloc metadata area. Using sucha returned PUMP or metadata processing value allows free to performdifferent conditional processing based on the whether the colorassociated with the memory location at address PTR2 denotes a validmalloc metadata color, such as red. Routine free may perform somerecovery or other action if PTR2 identifies an invalid malloc metadataarea as determined through the color of PTR2. Such action may be moredesirable than having the user code aborted due to a rule violation,trap, interrupt or other execution error.

In at least one embodiment using the RISC-V instruction set, toimplement returning a metadata processing value, a new instruction, gmd(get-metadata-info), may be added to the RISC-V instruction set such as:

-   -   gmd R1, R2, R3        where

R1 contains the result value returned by the PUMP or metadataprocessing;

R2 contains the address PTR2 which is tagged with the color of thememory location having address PTR2; and

R3 is tagged with the valid color as expected for a valid mallocmetadata area.

Thus, R2 and R3 may be registers that are input or source operands, andR1 may be the register containing the result or output. In thisparticular example, R3tag may be red denoting the color of a validmalloc metadata area and R2tag may be blue. The rule invoked by the newinstruction may output a return value as a Boolean in this exampledenoting whether R2tag=R3tag where the foregoing Boolean result may bethe return value output by metadata rule processing (e.g., PUMP output)stored in register R1 accessible to free included in the address spaceof user executing code. It should be noted that R1 may be tagged withRtag as the result tag consistent with discussion elsewhere herein.

The following describes logical processing that may be performed by codeof free using a C-like pseudo code description with PTR1, PTR2, and OFF1as described above:

free (char *PTR1)    PTR2 = PTR1 − OFF1; /** PTR    if (IS_RED (PTR2))then       PTR2 points a validly colored malloc metadata area. Performprocessing to       deallocate.    else       PTR2 does not point to avalidly colored malloc metadata area. Perform recovery processing.

In the above logical processing, IS RED may check to see if PTR2 is thecolor RED.

The recovery processing performed by code of the above-noted else blockmay, for example, try to locate the beginning of a valid malloc metadataarea by searching backward or forward from PTR2. Code of the above-notedelse block may allow termination of user code in a more defined anexpected manner such as with a runtime error message/condition denotingthe invalidly colored pointer PTR2.

The new instruction Get metadata info R1, R2, R3 may be included ininstructions generated, for example, as a result of compilation andlinking code of a free routine written in C to perform the above-notedlogical processing. An embodiment may want to control or restrict whatparticular code portions may be allowed to execute this new instruction.PUMP rules may be used to mediate or restrict when this new instructionis allowed to be executed by what routine. For example, code of free ormalloc may be allowed to execute the new Get metadata info instructionbut not user code. Any suitable technique, some of which are describedherein, may be used to provide the routine free with the neededprivilege or authority to execute the new instruction returning a PUMPvalue. For example, code of free may be tagged with a specialinstruction tag denoting that free is allowed to execute the newinstruction. For example, a loader may tag the new instruction appearingin free code with a special tag NI. Rules may be used to mediate orrestrict what code may be allowed to invoke the new instruction to thosehaving an instruction tag (CI tag) of NI.

Referring to FIG. 69, shown is an example 1210 illustrating inputs andoutputs of metadata rule processing in an embodiment in accordance withtechniques herein. Element 1212 may generally denote metadata processingas described herein. Inputs 1212 a to metadata processing may include,for example, the various tags and opcode information as describedherein. Outputs 1214 generated by metadata processing 1212 may includethe Rtag 1214 a and PCtag 1214 b as described elsewhere herein.Additionally, metadata processing may generate a new output that is thereturn value 1214 c. The return value 1214 c may be placed in aregister, such as R1 denoted above with the new instruction, which is inthe set of registers accessible to user process space/code execution.Consistent with description elsewhere herein, 1214 a and 1214 b denotetags which are placed, respectively, on a result (e.g., result registeror memory location) and the PC, whereby 1214 a-b are not accessible touser process space/code execution. It should be noted that whethermetadata processing returns the return value 1214 c may be conditionalon the particular instruction or opcode. For example, as describedelsewhere herein, the metadata processing outputs may be filtered asdescribed in connection with FIGS. 27-33 based on opcode using amultiplexer to enable/disable outputting return value 1214 c. Value 1214c in this example denotes a logical result of whether or not R2tag=R3tagwhen the opcode is that of the new instruction. Otherwise, if the opcodedoes not denote the new instruction opcode, a default value may beconditionally returned by metadata processing as the return value 1214c.

Referring to FIG. 70, shown is an example 1220 illustrating componentsand processing that may be performed in an embodiment in accordance withtechniques herein when returning a value by metadata processing to theuser execution domain, such as when executing the new instructionincluded in code of free as described above. For simplicity ofillustration, the example 1220 illustrates logic and components ofmetadata processing employed only with the destination or resultregister R1 for this new return value and the associated resultregister. Element 1222 a may generally represent PUMP inputs (e.g., tagssuch as the R2tag and R3tag in this example, opcode) as describedelsewhere herein for metadata processing. PUMP 1222 may include rulesfor the new instruction which check if the code tag is NI, and outputs alogical result denoting whether R2tag=R3tag (e.g., OP1 denoting thefirst input source operand R2 and OP2 denoting the second input sourceoperand R3 in this example). The rule results in outputting theforegoing logical result 1221 a. Element 1225 may denote a multiplexerwith the opcode used as the selector 1225 a for the multiplexer 1225.When the opcode of the current instruction denotes the particular opcodefor the new instruction Get metadata info, 1225 a results in selecting1221 a to be output as the return value 1214 c. Otherwise, if the opcodeis not that of the new instruction, 1225 a results in selecting adefault return value 1222 a as the return value 1214 c. The return value1214 c is a PUMP output stored in the destination register, RD, 1228(e.g., 1214 c is stored in D1 1228 b denoting the contents stored inregister RD accessible to code executing in the user process addressspace). Since RD 1228 is the result register, the rule may also resultin tagging RD with Rtag (e.g., Rtag is stored in tag portion T1 1228 awhere T1 is the tag word of the RD register). In at least oneembodiment, Rtag may be a special tag SPEC1 denoting that RD containsthe output of the new instruction. Based on symbolic logic as describedelsewhere herein where tag inputs to the rule are (PCtag, Cltag, OP1tag,OP2tag, MR tag) and rule outputs are (PCtag, Rtag) along with a thirdoutput, NEWOUT denoting the new return value 1214 c, the rule may beexpressed as:

-   -   gmd: (-, NI, t1, t2, -)→(-, SPEC1, NEWOUT)        where NEWOUT=1 if t1=t2 and NEWOUT=0 otherwise.

More generally, the foregoing use of a new instruction may be used in anembodiment in accordance with techniques herein to return a value thatis any suitable and desirable value that may be used by code speciallytagged (e.g., with NI) to denote those occurrences of the newinstruction that are allowable via invoked metadata processing rule(s).

Alternate embodiments may avoid adding a new instruction. This can bedone by code-tagging an existing instruction to control this behaviorand setting the care-bit to select the value output in this case.Another alternative may add a value-output-care-bit that is also anoutput of the PUMP so that the rule can determine the cases in which thevalue output should flow to the RD value result. This second case allowsthe opcode to behave normally when not tagged, and only exhibit thisspecial behavior when given the appropriate code tag.

What will now be described are techniques that may be used to guaranteethat a particular sequence of instructions is performed atomically as asingle unit or complete sequence in a specified order from the firstinstruction to the last instruction of the sequence. Additionally, suchtechniques guarantee that there is no transfer of control into thesequence of instructions other than to the first instruction of thesequence and that there is no transfer or exit out of the sequence otherthan through the last specified instruction of the sequence. Forexample, consider the simple instruction sequence of FIG. 71.

In the example 1400, shown is a sequence of 2 instructions 1402 and1404. The first instruction 1402 reads or loads contents from a memorylocation (where the address of the memory location is stored in R2) toR1. The second instruction writes or stores a zero (0) to the samememory location (the memory location having the address stored in R2).Such an instruction sequence may provide for ensuring that a value readfrom a memory location (having the address specified in R2) is used onlyonce whereby the old value is erased or zeroed out from the memorylocation immediately after the value is read from the memory location.Thus, the zeroing out of the memory location is performed by the secondinstruction 1404 of the sequence and is required to be performed as thenext instruction in the sequence after the first instruction 1402 thatreads the value from the memory location.

In at least one embodiment in accordance with techniques herein using aRISC architecture, rules may be used to enforce the foregoing linearityof a data item and atomicity of the instruction sequence of 1400. Insuch an embodiment, the PC tag (PC new tag) may be updated tocommunicate the state of the sequence of the next expected instructionin the sequence. In at least one embodiment, one solution is to tag theinstruction 1402 with a CI tag denoting it as a linear read instruction.Additionally, (R2), denoting the memory location having the addressstored in R2, may be typed and tagged as a linear variable with a uniquemetadata id X1 (e.g., X1 uniquely identifies this linear variable fromall other linear variables). A first rule may be triggered as a resultof the first instruction 1402. The first rule may indicate that onlyinstructions tagged as linear reads are allowed to read from linearvariables. Additionally, the resulting PCnew tag may beclear-linear-variable-X1-next to denote that the next instructionexecuted needs to clear the linear variable X1. A second rule may betriggered as a result of executing the second instruction 1404 where thevalue of the operand, zero (written to the memory location), is taggedwith a special EMPTY tag denoting the special value used to initializeor clear the memory location. Additionally, the memory location isrequired to be the linear variable X1 denoting the particular taggedlinear variable from the immediately preceding instruction 1402. If thesecond instruction following 1402 does anything other than write anEMPTY value to linear variable X1, a trap is caused. Thus the secondrule enforces the desired sequentially and atomicity of the sequence ofinstructions in 1400.

More specifically, assume the first instruction 1402 is a loadinstruction that loads into R1 the contents from the memory locationhaving the address stored in R2. The load instruction may be as follows:

-   -   load R1, (R2)        Additionally, assume the second instruction 1404 is a move        instruction that move zero (0) into the memory location having        the address stored in R2. The move instruction may be as        follows:    -   move 0, (R2)        Following conventions noted elsewhere herein a rule may be        defined as an opcode, input tags—PCtag, CItag, OP1tag, OP2tag,        Mtag—and output tags—PCnewtag, Rtag. Based on the foregoing rule        conventions, for the first load instruction, OP1 is R1, OP2 is        R2 and (R2) is the memory location tagged as Mtag. The first        rule triggered by the first load instruction may be:    -   load: (-, linear read, -, -, linear variable        X1)→(clear-linear-variable-X1-next, -)        Based on the foregoing rule conventions, for the second store        instruction, OP1 is 0, OP2 is R2 and (R2) is the memory location        tagged as Mtag. The second rule triggered by the second move        instruction may be:    -   move: (clear-linear-variable-X1-next, -, EMPTY, -, linear        variable X1)→(default tag, -)

This example shows how tags and rules can be used to guarantee theindivisibility of a particular instruction sequence. One skilled in theart can readily see that this general technique can be applied in manyother scenarios where it is desirable to enforce that data may only beaccessed in a particular way as part of a specific sequence ofinstructions. One skilled in the art can also readily see that thistechnique can be adopted for any case where strict enforcement of aninstruction sequence is required. As noted above, the general techniqueinvolves tagging the new PC (e.g., PCnew tag) from instruction N in thesequence with a special tag that is checked as the PC tag in a ruletriggered by the next expected instruction N+1 in the glued sequence.

What will now be described are techniques that may be performed as partof booting or starting up a system in an embodiment in accordance withtechniques herein based on the RISC-V architecture. Following paragraphsmay refer to various CSRs described elsewhere herein such as inconnection with the example 900 where such CSRs may be used inconnection with the metadata processing domain.

As described elsewhere herein, a bootstrap tag may be hardwired or avalue stored in a particular ROM location. As part of booting thesystem, a segment of bootstrapping code may be executed that generallyperforms initialization including initializing different CSRs, memoryand the like. As part of initializing, such processing also initiallytags memory locations with a default tag value derived from an initialbootstrap tag. In at least one embodiment, a CSR such as the boottag CSR(e.g., sboottag CSR as in the example 900) may be initialized with aspecial bootstrap tag used as the initial “seed” tag from which allother tags in the system are derived. Different code entities, such as aloader, may have their instructions specially tagged (e.g., CI tags setto special instruction tag) to thereby designate the loader as havingparticular privileges or authority to perform tasks that other code nothaving the special instruction tag is not allowed to do. The foregoingmay be enforced using rules triggered by code of the loader that examinethe CI tag to ensure it is the special tag in order to have thetriggered rule perform a desired tagging operation. Thus, for example,the special CI tag used to tag instructions of the loader may begenerated or derived from the bootstrap tag as a result of special rulestriggered by executing code as part of the startup process. Generally,once some portion of code or stored instructions are tagged, rules maybe triggered by execution of such tagged code to generate more desiredtags and also place such generated tags on code and data. The foregoingand other aspects are described in more detail below.

At startup or booting of the system, the tag mode such as stored in thetagmode CSR (e.g., 901 r of the example 900) may initially be off (e.g.,911 a of the example 910). A bootstrap ROM program may be executed thatfirst directly sets the default tag CSR (e.g., 901 c of the example 900)to a special default tag value. Subsequently, the bootstrap program mayset the tagmode CSR to a mode whereby the metadata processing domainwrites the default tag as stored in the default tag CSR on all results.In other words, while in defaulttag tag mode (e.g., 911 b of the example900), the PUMP output Rtag is always the default tag value.

Subsequently, after memory locations have been initialized and taggedwith the default tag, processing may be performed to generate an initialset of tags that will be used to further generate or derive all othersubsequent tags (e.g., the initial set may be further used to derive oneor more other generations of tags in an unbounded manner). Suchprocessing may include executing an instruction sequence or code segmentthat triggers rules to generate the initial set of tags. In this case,the tag mode may be set to an appropriate tag mode level that engagesthe PUMP during execution of the code segment. For example withreference to the example 910 if boot code is executing in hypervisormode, the tag mode may be set to either x110 as denoted by 911 e or x111as denoted by 911 f to engage the PUMP during execution of the codesegment whereby rules are triggered and enforced as a result of the codesegment instructions.

It should be noted that prior to executing the above-noted code segment,processing may be performed to verify or validate the code segment. Forexample, in at least one embodiment, the above-noted code segment may bestored in an encrypted form where, prior to executing, the code segmentis decrypted and verified or validated (e.g., such as using a digitalsignature) to ensure the code segment has not been tampered with ormodified.

To further illustrate, the bootstrap program may include in theabove-noted code segment 4 instructions executed while the PUMP isengaged to thereby generate an initial set of tags:

-   -   1. R1←read boottag CSR    -   2. Add R2←R1+1    -   3. Add R3←R2+1    -   4. Add R4←R3+1

In instruction 1 above, R1 is a general-purpose register. Instruction 1reads the boottag CSR, transferring both the value stored in the boottagCSR and the tag stored in the boottag CSR onto R1. The boottag CSR waseither set to hold a particular tag during processor reset or byprivileged mode write of the CSR, including its tag. The read from theboottag CSR may also clear the boottag CSR so that it is not availableto be retrieved after this initial retrieval during boot.

In each of the add instructions forming instructions 2-4 of the above ofthe form “Add Rn←Ry+1” wherein Rn denotes the target or result registerto store the results of the Add, and wherein Ry also denotes a registera source operand. Instruction 2 of the foregoing code segment maytrigger a second rule that generates a second tag from the first tag andplaces the second tag on the memory location pointed to by R2.Instruction 3 of the foregoing code segment may trigger a third rulethat further generates a third tag from the second tag and places thethird tag on the memory location pointed to by R3. Instruction 4 of theforegoing code segment may trigger a fourth rule that further generatesa fourth tag from the third tag and places the fourth tag on the memorylocation pointed to R4. In this manner the foregoing code segment may beused to generate an initial set of 4 tags stored as tag values onregisters. The foregoing general technique may be further extended in asimilar manner to generate any desired number of tags of the initialset.

Generally, in generating an initial set of tags in at least oneembodiment, the particular number of tags in the initial set may be apredefined number. Each of the special tags may be generated as a resultof a different unique rule triggered when executing an instruction. Eachinstruction, such as in the code segment above, may result in a cachemiss and thereby result in execution of the cache miss handler tocalculate the Rtag as part of the rule outputs for the particularinstruction, where Rtag is one of the tags of the initial set. In amanner similar to the instructions of the code segment above, adifferent code sequence may be executed at different points in time tofurther generate other tags using one of the tags of the initial set.Thus, each tag in the initial set may denote a tag generator used tofurther generate another sequence of tags. In the foregoing example, theAdd instruction may be used in generating a next tag generator that maybe used to generate another entire sequence of tags. As discussed below,a tag generator of the initial set (which is itself a further taggenerator used as a starting point to generate another sequence) may bedistinguished from a regular or non-generating tag which cannot befurther used as a generator to generate another sequence of tags. Thus,particular instructions such as ADD may be used to trigger rules andmiss handling to generate a set or sequence of tag generators. This maybe contrasted with another instruction, such as MOVE, which may triggerrules and miss handling to generate a non-generating tag in a sequence.In connection with code such as malloc, an ADD instruction may besimilarly used to generate a new application tag color generator used togenerate a sequence of different colors for a first application (e.g.,new application tag color generator may be APP1 used to generate asequence of different colors RED-APP1, BLUE-APP1, GREEN-APP1, etc forthe particular application). A tagged ADD instruction may then be usedto obtain the next tag in the particular applications specific sequence,such as one of RED-APP1-gen, BLUE-APP1-gen, or GREEN-APP1-gen. Then atagged MOVE instruction may be used to generate the actual colors,RED-APP1, BLUE-APP1, or GREAN-APP1 from RED-APP1-gen, BLUE-APP1-gen, orGREEN-APP1-gen, respectively, (where RED-APP1, BLUE APP1, GREEN-APP1cannot be used to further generate additional tag sequences).

The code segment of the bootstrap program that is executed while thePUMP is engaged may also include additional code that, when executed,triggers rules to tag kernel code/instructions and additionally tagother code modules or entities with any desired special instruction tagsto enable such specially tagged code to have desired privileges orcapabilities. For example, the code segment may include instructionsthat trigger rules to tag the loader code, and code of routines mallocand free with special instruction tags extending privileges or authorityto such code to perform privileged tagging operations. The special codetags may be generated from the initial set of tags in a manner similarto that as noted above using a predetermined code sequence/set ofinstructions that triggers rules to generate further desired tags andalso appropriately tag additional code and/or data with the generatedtags.

In at least one embodiment, additional measures or techniques may betaken in connection with portions of the above-noted code segment. Forexample, the above noted 4 instructions used to generate the initial setof tags may be included in a first instruction sequence using rules of a“glue” policy to enforce sequentiality and atomicity such as describedelsewhere herein (e.g., example 1400).

After the code segment noted above has been executed to generate theinitial set of tags and further specially tag kernel code and any otherdesired instructions, control may be transferred to additional bootcode. In at least one embodiment based on the RISC-V architecture, theadditional boot code may be executed at a hypervisor privilege level.Such additional boot code may, for example, include instructionstriggering loading of an initial set of rules into the PUMP. Oncebooting has been completed, the PUMP tag mode as denoted by the tagmodeCSR may be set to suitable level to engage the PUMP in connection withuser code such as executes at the user privilege level (e.g., set tagmode as in 911 c of the example 910 to denote PUMP is engaged andoperations in U (user) mode or privilege level only).

Referring to FIG. 72, shown is a flowchart of processing steps that maybe performed in an embodiment in accordance with techniques herein. Theflowchart 1600 summarizes processing described above. At step 1602, tagmode is set to off where the tagmode CSR denotes the PUMP off state asdescribed elsewhere herein in connection with 911 a of the example 910.At step 1604, the boottag CSR is initialized to the special bootstraptag. At step 1606, execution of the bootstrap program is commenced. Atstep 1608, the bootstrap program may set the defaulttag CSR to thedefault tag. At step 1610, the tagmode CSR may be modified to a modethat writes the default tag on all results (e.g., each Rtag=default tagwhile in this tag mode). At step 1612, instructions may be executed thattrigger rules to initialize memory locations and tag the memorylocations with the default tag. At step 1614, the tagmode CSR may bechanged to a mode that engages the PUMP during execution of subsequentcode segment in step 1616. At step 1616, the subsequent code segment isexecuted with the PUMP engaged. The code segment includes instructionsthat trigger rules to generate an initial set of tags, clear boottagCSR, tag kernel code, and tag additional code portions with special codetags providing such tagged code with extended capabilities, authoritiesand privileges as desired. At step 1618, control may be transferred toadditional boot code that is executed. When the boot process iscomplete, the system is now ready to execute user code with the PUMPengaged and operational for executing user code.

What will now be described in more detail is how to generate tags fromthe bootstrap tag. The tag generation processing commencing with thebootstrap tag may also be referred to as a tag tree or tree of life.More generally, the tag generation process forms a hierarchicalstructure as illustrated in the example 1620 of FIG. 73.

The example 1620 illustrates the boottag 1621 as the root of the taggeneration process. Element 1621 a-d may denote the initial set of tagssuch as generated as described above. In this example, the initial setof tags 1621 a-d may include an initial OS special instruction tag 1621a used to further generate a sequence 1622 of an unbounded number ofspecial instruction tags which may then be applied 1623 to taginstructions of different code portions or modules 1624. From theinitial OS special instruction tag 1621 a, additional tags 1622 may begenerated for the different modules to be tagged. For example, a firstOS special instruction tag1 1622 a may be generated for malloc which isapplied 1623 a to malloc code whereby instructions of malloc are tagged1624 a with the special instruction tag 1 1622 a. In this manner, malloccode may be tagged with a special instruction tag identifying malloc asa tag generator (e.g, denoting that malloc code has privileges tofurther generator other new tags and further use the newly generatedtags to tag other memory cells).

In this example regarding malloc, 1621 b may be the initial malloc tagused to further generate malloc tag generator application tags 1626, oneper user application since an instance of malloc is included in eachuser application. We want to give each such malloc instance in each userapplication the privilege to generate different colored tags as includedin 1625.

Generally, the example 1620 illustrates an initial set of tags 1621 a-dfor Special Instruction tags 1621 a, Malloc 1621 b, CFI 1621 c, andTaint 1621 d. Thus, each of the tags 1621 a-d in the vertical display oftags in the first row (other than boottag 1621) denotes a differentinitial tag used for a generating an unbounded tag sequence. Forexample, the value 1621 a is used in further deriving or generating anunbounded number of special instruction tags 1622. The value 1621 b isused in further deriving or generating an unbounded number of values1626. Each instance of 1626 may be further used as a generator ofanother unbounded sequence of tags for each application. For example,1626 a denotes a generator value used to further generate anotherunbounded sequence 1629 of different colors used for a singleapplication appl. In a similar manner, each different generator value of1626 may be used to further generate an unbounded number of colors foreach application.

The value 1621 c may be used as a generator in further generating anunbounded number of values 1627. Element 1627 is similar to 1626 in thateach occurrence of CFI tag generator n for a particular application orapp N denotes a privilege or ability to further generate anotherunbounded sequence. For example, 1627 a denotes a generator value usedto further generate another unbounded sequence 1630 of different colorsused for a single application app 1. In a similar manner, each differentgenerator value of 1627 may be used to further generate an unboundednumber of colors for each application.

The value 1621 d may be used as a generator in further generating anunbounded number of values 1628. Element 1628 is similar to 1626 and1627 in that each occurrence of a tag generator n for a particularapplication or app N denotes a privilege or ability to further generateanother unbounded sequence. For example, 1628 a denotes a generatorvalue used to further generate another unbounded sequence 1631 ofdifferent colors used for a single application app 1. In a similarmanner, each different generator value of 1628 may be used to furthergenerate an unbounded number of colors for each application.

As illustrated sequences or subtrees for CFI and Taint originating,respectively, from 1621 c-d are similar to the Malloc subtreeoriginating from 1621 b. In the example 1620, nxtTag or TlnxtTag is usedto denote a next element in a generated unbounded sequence, and getTagto extract a next tag from a sequence member. Generally, getTag may beused to denote extracting a tag to use which is, itself, not a taggenerator. If the usable tag is going to be given to a particular codeportion to use, we don't to want to also give the code portion theability to generate tags. For example, we want to give each applicationa Malloc Tag Generator for that application (e.g. ApplColorTagX), but donot want to give the application the ability to generate the Malloc TagGenerator for other applications. So, getTag changes the type fromgenerator to instance. The distinction between nxtTag and TlnxtTag isthat nxtTag is usable without a “tagged instruction”, but TInxtTag isone that is only usable by a suitably tagged instruction.

The Malloc Application Tag sequence 1626 allows the operating system orloader to generate Color Tag generators for each application. Forexample, element 1626 a denotes an application specific color taggenerator value used to generate tags of the application color sequence1629. Within an application, the AppYColorTag sequence 1629 allowsmalloc to generate an Authority for each color. That color authority canbe used to: color the cells for allocated memory, color a pointer forthe allocation and free cells of that color (e.g., when free isinvoked). The use of colors such as with malloc and free are describedelsewhere herein.

In this manner, different tags may be reserved for different uses. Frominitially tagged kernel instructions as noted above, kernel code may beexecuted that further tags other code portions with differentcapabilities or authority. For example, kernel code of the operatingsystem may further tag other code entities, such as a loader, withspecial privileges such as granting the loader the ability to furthertag other code and data, generate additional tag generators, and thelike. The loader when loading a user program including malloc mayfurther tag malloc code with special instruction tag(s) denoting it asmalloc code giving it the capability to further generate other tags usedto color different memory regions. A particular instruction tag placedon code of the loader thus provides the loader with one set ofprivileges. Placement of a second different instruction tag on malloccode provides malloc code with another different set of privileges.Generally, when performing tag generation of a sequence a current tag inthe sequence is saved as state information which is referenced and usedin connection with generating a next tag in the sequence. As describedherein, such state information regarding the current tag in the sequencemay be saved and used in the metadata processing domain. The currenttag, or more generally metadata processing state information, may besaved and restored as a result of rule processing and cache missprocessing. The current tag in a sequence, such as the last colorallocated for use for a particular application, may be saved as acurrent state of the sequence as a tag on a specified memory location.When a new next color for the application needs to be allocated, code ofmalloc may trigger rules which retrieve the last allocated color for theapplication and use the last allocated color to determine the next colorin the application-specific color sequence. Generally, generating aunique sequence of tags may include executing instructions that triggerrules that perform the following:

-   -   1. storing/saving sequence state in a tag portion of an atom        (e.g., register, memory location);    -   2. executing an instruction that triggers a rule which generates        the next tag of the sequence using the saved/stored sequence        state; and    -   3. storing/saving the next tag of the sequence (generated from        2) in the tag portion of an atom where the next tag is now the        updated current state of the sequence.

With reference back to the example 1620, the loader may allocate foreach application using malloc a particular one of the malloc taggenerator application tags of 1626. The loader may, for example, executecode triggering a rule which generates the next malloc tag generatortag, such as 1626 a, and then stores this tag as state information viatagging a memory location. Subsequently, on a first call to malloc bythe application, code of malloc may execute that triggers a rule whichthen retrieves the saved malloc tag generator tag, uses the saved tag togenerate the first color for the application, and then updates the savedstate information to store the first color as the last or most recentcolor generated for the application. On a second call to malloc by theapplication, code of malloc may execute that triggers a rule which thenretrieves the previously saved first color, uses the saved first colorto generate a second color for the application, and then updates thesaved state information to now store the second color as the last ormost recent color generated for the application. In a similar manner,other subsequent calls to malloc may be trigger other rules to allocateadditional colors based on the saved state information (e.g., mostrecently allocated color) for the application.

What will now be described are aspects of a direct memory access (DMA)architecture that may be included in an embodiment in accordance withtechnique herein. Generally, described in following paragraphs is use ofan I/O PUMP to mediate DMAs issued from a source, such as an untrusteddevice connected to a first interconnect fabric that uses untagged data,to access data stored in memory of a second interconnect fabric thatuses tagged data.

Referring to FIG. 74, shown is an example of components that may beincluded in an embodiment in accordance with techniques herein. Theexample 1500 includes components similarly numbered to those of theexample 700 and others (e.g., FIGS. 57-60) described elsewhere herein.Additionally, the example 1500 also includes I/O PUMP 1502 andadditional actors, DMA request sources or initiators 1504 a-c that mayissue DMA requests to access data stored in memory 712 c. The example1500 includes Ethernet DMA device A 1504 a, Ethernet DMA device B 1504b, and UART (universal asynchronous receiver/transmitter) or serialcommunications device 1504 c connected to the untagged fabric 715. A DMArequest to read or write data may originate from one of the devices 1504a-c. The request is sent to the I/O PUMP 1502 which performs processingto determine whether the DMA request is allowed and if so, allows therequest to proceed. Thus, the I/O PUMP 1502 may be characterized asmediating DMA requests received from over the untagged fabric 715whereby the general assumption is that devices connected to 715 issuingsuch DMA requests may be untrusted.

In at least one embodiment, the I/O PUMP 1502 may be an instantiation ofthe PUMP as described herein (e.g., FIG. 22) with a difference that therules enforced are those of a DMA policy controlling DMA access intomemory 1712 c. The foregoing use of the I/O PUMP 1502 is line with thegeneral architecture of assuring that all instructions, including memoryoperations, are mediated by rules. If autonomous DMA devices 1504 a-cwere allowed direct, unmediated access to memory, the DMA devices 1504a-c may undermine the invariants and safety properties that the rulesare enforcing. Consequently, to allow DMA, an embodiment in accordancewith techniques herein may also enforce rules on DMA access into thememory 712 c. Analogous to the PUMP that enforces rules for processorinstructions, the I/O PUMP 1502 enforces rules for memory loads andstores from DMA devices, such as 1504 a-c. Generally, the I/O PUMPmediates all loads and stores. In at least one embodiment describedherein based on the RISC-V architecture, the I/O PUMP uses CSRs andperforms rule cache miss handling in a manner similar to that asdescribed elsewhere herein in connection with the PUMP used in a RISC-Varchitecture. The I/O PUMP 1502 has a set of CSRs similar to the PUMP,but accesses them via memory mapped addresses. Accesses to I/O PUMP CSRssuch as described in following paragraph in connection with the example1520 may also be tag protected using rules. Rule cache missesencountered when attempting to locate a rule in the I/O PUMP trigger aninterrupt to be serviced by the processor, RISC-V CPU 702. The I/O PUMPuses the same rule resolution process as the processor 702 but there isa single DMA policy including only rules for DMA loads and stores toaccess data in memory 712 c. The I/O PUMP writes atomically into memory712 c (e.g., writes the tag and value as a single atomic operation).However, in some embodiments, the complete process from reading the Mtagto writing the Mtag (e.g., processing to perform a tag check or validateand write) may not be atomically with a standard store.

The I/O PUMP 1502 is a rule cache for SDMP. The I/O PUMP provides amapping between a set of tags involved in a DMA operation and the resultof the operation. In at least one embodiment, the I/O PUMP runsindependent of the processor 702. Since the I/O PUMP 1502 is a cache, itwill take misses when it has never seen a set of inputs before(compulsory) or when it was unable to hold onto a rule (capacity, orperhaps conflict). This results in a rule cache miss with respect to theI/O PUMP in a manner similar to rule cache misses as described hereinfor the PUMP. Misses with respect to the I/O PUMP rule cache 1502 raisean interrupt that is then handled by in software by a rule cache misshandler system—the same one that services processor 702 miss traps. On arule miss with respect to the I/O PUMP 1502, inputs are communicated tothe Miss Handler (such as executed on code of the processor 702 in themetadata processing domain) through I/OPUMP CSRs described below (e.g.,example 1520), and rule insertion is provided back to the I/O PUMPthrough CSRs. I/OPUMP misses cause the I/O PUMP to be disable untilserviced by the processor 702. In at least one embodiment, the disabledstate of the I/O PUMP means all DMA transfers mediated by the I/O PUMPare stalled until the I/O PUMP miss is serviced.

Consistent with discussion elsewhere herein with the PUMP, I/O PUMPinputs include an opgroup (opgrp), tags for the DMA instruction and itsoperands (e.g. PCtag, CI tag, OP1 tag, OP2 tag, Mtag (also referred tosometimes herein as the MRtag). I/O PUMP outputs may include the Rtagand PCnew tag (tag for the PC of the next instruction) as describedherein. In connection with the I/O PUMP, such inputs and outputs mayhave further meaning and values as described below in one embodiment.

Following are I/O PUMP inputs in one embodiment:

-   1. Opgrp—there are current two: load and store-   2. PCtag—state of the DMA IO device (analog to the PCtag for code)-   3. Cltag—tag identifying the DMA IO device (analogous to instruction    tags on a designated region of code)-   4. OP1tag—assume always “public, untrusted” (not physically    represented in IOPUMP cache, but used for rules)-   5. OP2tag—same as OP1tag-   6. Mtag—tag on memory input to DMA operation-   7. byteenable—which bytes are being read/written?    Following are I/O PUMP outputs in one embodiment:-   8. Rtag—tag on memory result for a store-   10. PCnew tag—state of the DMA I/O device after this operation

With the I/O PUMP, there may be no programmable opgroup mapping table(e.g., example 420). Rather, an opgroup used by the I/O PUMP to look upa rule may be a fixed opcode denoting a single opgroup for DMA load andDMA store operations. In at least one embodiment, there is no caremasking for the I/O PUMP.

When there is a rule cache miss in connection with the PUMP as describedherein such as in FIG. 22, it may be expected that the processor 702will automatically reissue the instruction that caused the miss afterits corresponding rule has been inserted into the PUMP rule cache. As aresult, rule insertion simply places the rule in the PUMP cache andexpects the instruction to be re-issued in order to get the taggedresult. However, behavior with DMA operations varies from the foregoing.DMA operations are not expected to be interrupted and to require retryoperations. In order to support these DMA operations, rule insertion maybe handled differently for the I/O PUMP. In particular, once the I/OPUMP has faulted due to a miss, processing may hold the pending DMAoperation and wait for the processor 702 (e.g., performing rule misshandling to calculate the output tag Rtag and PC new tag for the newrule) to supply the missing output tags for the rule (assuming it willbe allowed). When the outputs are supplied, in addition to triggering arule write into the IOPUMP, the outputs are forwarded to the DMApipeline (e.g., described in connection with example 1540 below) just asif they had come from the I/O PUMP so the operation can continue withoutforcing the operation to be re-issued to the I/O PUMP. Rule violationsmay be handled by supplying a designated disabled-DMA-device tag for theupdated PCtag, PCnew tag, that will signal that the operation is notallowed and no further DMA operations will be allowed from thatparticular DMA device 1504 a-c until its PCtag is reset. Generallydevice tags for a particular DMA device, such as one of 1504 a-c,issuing a DMA operation or request may be the particular values of theCI uniquely identifying the issuing DMA device (e.g. source of the DMArequest), and the PC tag denoting the current state of the DMA device.In at least one embodiment, the PC tag may be set to a particular valueat a point in time disabling further processing of DMA requests from theparticular DMA device identified by the CI tag.

Referring to FIG. 75, shown is a table of CSRs that may be used in bythe I/O PUMP in an embodiment in accordance with techniques herein. Thetable 1520 includes an address column 1524 (denoting the memory mappedaddress of a CSR), a name column 1526 and a description column 1528.Each row of the table 1520 corresponds to one of the defined CSRs usedby the I/O PUMP. Row 1522 a indicates that CSR transaction id hasaddress 0x00. A write to the transaction id CSR increments the currenttransaction id stored (e.g., for prefetch) and reading from thetransaction id CSR return the current transaction id stored in thetransaction id CSR. Row 1522 b indicates that CSR opgrp has address0x01. The opgrp CSR contains the opgroup for the current DMA instructionand is used on a rule miss as an input to the rule miss handler. Row1522 c indicates that CSR byteenable has address 0x02. The byteenableCSR indicates which of the bytes in a word the DMA operation effects andis used on a rule miss as an input to the rule miss handler. Consistentwith other discussion herein, this allows policies to provide byte-levelprotection; a rule triggered may check to ensure that the bytes of theDMA requested data are allowed to be accessed by a particular DMA deviceinitiating the request such as by specially tagging memory portionsaccessible to the different DMA devices. Row 1522 d indicates that CSRpctag has address 0x03. The pctag CSR contains the PC tag for thecurrent DMA instruction and is used on a rule miss as an input to therule miss handler. Row 1522 e indicates that CSR citag has address 0x04.The citag CSR contains the CI tag for the current DMA instruction and isused on a rule miss as an input to the rule miss handler. Row 1522 findicates that CSR mtag has address 0x07. The mtag CSR contains the Mtag for the current DMA instruction and is used on a rule miss as aninput to the rule miss handler. Row 1522 g indicates that CSR newpctaghas address 0x08. The newpctag CSR contains the PC new tag placed on thePC after completion of the current DMA instruction (e.g. output of PUMPand cache miss handling). Row 1522 h indicates that CSR rtag has address0x09. The rtag CSR contains the tag placed on the memory result of thecurrent DMA instruction (e.g. output of PUMP and cache miss handling).Row 1522 i indicates that CSR commit has address 0x0A. Writing to thecommit CSR results in a comparison between the value written to thecommit CSR and the current transaction id (as stored in the transactionid CSR). If the foregoing two match, the match triggers a write of arule to the I/O PUMP. The rule written includes an opcode and tag inputsand outputs (as determined by miss handling) for the current DMAinstruction. Row 1522 j indicates that CSR status has address 0x0E. Thestatus CSR contains a value denoting a status of the I/O PUMP. Forexample in one embodiment as described herein, the status CSR may denotewhether the I/O PUMP is enabled or disabled. It may be disabled in thecase of a PUMP I/O rule cache miss as described elsewhere herein. Row1522 k indicates that CSR flush has address 0x0F. The flush CSR, whenwritten to, triggers a flush of the I/O PUMP (e.g., flushes or clearsrules from the I/O PUMP cache).

In at least one embodiment, if bit 0 of the status CSR is 1, it meansthe I/O PUMP is disabled and if the bit 0 otherwise has a value of 0, itmeans the I/O PUMP is disabled. PUMP I/O misses disable the pump. Bit 1of the status CSR indicates whether the PUMP has faulted and is waitingfor service (e.g., Bit1=1 implies I/O PUMP faults/cache miss and waitingfor service). Bit 2 of the status CSR indicates whether an I/O PUMP rulemiss is currently being resolved by a rule cache miss handler and, ifthe transaction id matches, will provide the inserted results directlyto the pending miss operation. All the foregoing bits of the status CSRare reset (e.g., bit 0=enabled, bit 1=no fault, bit 2=no pending miss)by a commit operation (successful or unsuccessful). Writing to thestatus CSR may also be performed to reset the foregoing bits, forexample, as needed on startup to initially enable the I/O PUMP. Reset ofthe status CSR for an unsuccessful write allows the DMA device to retrythe operation and retrigger the fault.

Load/store memory operations by the processor 702 to the I/O PUMP CSRsshould be tagged with the iopump CI tag. Policy rules should be in placeto restrict operations to instructions having the iopump CI tag.Individual I/O PUMP CSRs do not have tags.

Each device 1504 a-c on the untagged or untrusted fabric 715 may beconfigured with its own tag that is presented as a device tag when theprocessor performs loads or stores to the device (e.g., see 1534 b wherethe device tag is stored in the device register file described below andspecified as the CI tag when the particular device performs a DMA loador store). This allows fine-grained control over which code andauthorities can access which devices directly. The same tag is presentedon all loads and stores to the device, and the tag does not change basedon load and store operations. The particular device tag associated with,and identifying, each device 1504 a-c may be stored in a device registerfile. A particular device tag specified for a device 1504 a-c may onlybe changed by modifying the device register file. The device registerfile may denote, for each device 1504 a-c, a unique target device id(used to identify the device on the untagged or untrusted fabric 715)and a target-device specific tag for the unique target device id. In atleast one embodiment, the device register file may itself be accessed asa device on the untrusted fabric 715 with its own device tag. Tobootstrap the use of the device register file, the device tag registerfile's own tag (stored in the device register file) may be written tothe file during startup before the PUMP is enabled. For example, thedevice tag register file's own tag may be written to the file as part ofboot processing while the PUMP is off (e.g., tagmode denoting by 911 aof the example 910). The CI tag of an instruction may identify thetarget id of the DMA target device performing a load or storeinstruction where the CI tag may be used in rules triggered by such loadand store operations to restrict (e.g., allow or not allow) a particularload or store operation by the specified DMA device. Additionally, if aparticular DMA device performs load and/or store operations which arenot allowed, a state associated with the particular DMA device may bemodified to disabled so that further requests (e.g., DMA loads andstores) are ignored.

As noted above, a DMA device that initiates or is a source of DMArequests or instructions may have an associated status indicated by thePCtag of the DMA device. In particular, a unique PCtag may be used todenote a disabled status with respect to DMA operations being allowedfrom a DMA device (identified by the CI tag). Disabled initiators havetheir DMA requests rejected at the start of the DMA or Trustbridgepipeline described below (e.g., examples 1530 and 1540).

It should be noted that an embodiment may have a single I/O PUMPmediating all DMA traffic, an I/O PUMP per DMA engine, or multiple I/OPUMPs that mediates DMA traffic for multiple DMA engines. Illustrated inthe example 1510 is a single I/O PUMP for a single DMA engine (e.g.,single memory 712 c). Use of a single I/O PUMP as in the example 1500may become a bottleneck and thus an embodiment may choose to havemultiple I/O PUMPs mediate I/O traffic. In such an embodiment wherethere are multiple I/O PUMPs, each may be enabled or disabledindependently so that even though a first portion of one or more of themultiple I/O PUMPs may be disabled (due to an I/O PUMP miss), theremaining second portion of the multiple I/O PUMPs may be enabled andcontinue to service DMA requests.

In at least one embodiment, different DMA devices acting as initiatorsor sources of the DMA operations may each be allowed to access onlyspecified portions of memory 712 c. Different portions of memory 712 caccessible via DMAs may each be tagged with a distinct tag. For example,device 1504 a may have access to a first range of addresses of memory712 c and device 1504 b may have access to a different second range ofaddresses of memory 712 c. Memory locations of 712 c corresponding thefirst range may be tagged with a first tag and memory locations of 712 ccorresponding to the second range may be tagged with a second tag. Inthis manner, rules may be used to enforce or restrict access of device1504 a to memory locations in the first range and enforce or restrictaccess of device 1504 b to memory location in the second range. As avariation, different tags may be associated with a type of allowedaccess (e.g., read only, write only read and write). In a similarmanner, in an embodiment having multiple DMA engines accessing the samememory 712 c, different portions of the single memory 712 c accessibleexclusively to each of the DMA engines may be uniquely tagged wherebyrules enforce or restrict access of each DMA engine to its specifiedaddress range of memory locations.

Referring to FIG. 76, shown is an example illustrating data flow betweentrusted fabric 1532 (e.g., corresponding to tagged interconnect fabric710) and untrusted fabric 1536 (e.g., corresponding to untaggedinterconnect fabric 715) in an embodiment in accordance with techniquesherein. Element 1534 generally represents processing performed by theI/O PUMP 1534 a in connection with DMA mediation between 1532 and 1536.Element 1534 may denote a trust bridge or DMA pipeline 1534 c performedto validate and service a DMA operation as part of the DMA mediation.Element 1538 a may denote the output channels from the untrusted fabric1536 (e.g., such as to the DMA devices 1504 a-c in the example 1500).Element 1538 b may denote the input channel to the untrusted fabric 1536(e.g., from one of the devices 1504 a-c). Generally, the I/O PUMP 1534 awill need to issue read requests during DMA read and write operations tovalidate that the tag on the target memory allows the requested DMAaccess. The I/O PUMP will need to buffer requests (as described below inthe example 1540 between processing stages) and perform master controlof tagged communication operations.

Element 1537 denotes values provided as inputs to load (or retrieve) theI/O PUMP CSRs as described in the example 1520. Additionally devicestate information for the different DMA device initiators may be storedin the untrusted fabric device register file 1534 b including the PCtag(e.g., state of the DMA device such as whether requests from this DMAdevice are disabled) and CItag (e.g., DMA device unique identifier) forthe DMA device initiator (e.g., such as 1504 a-c on the untrusted fabric715). The entry in the device register file 1534 b for a particular DMAdevice performing a DMA load or store may provide the CI tag and PCtagvalues for the current DMA load or store. Element 1535 a may denote thechannel used for devices on the untrusted fabric 1536 to make mediatedDMA processing requests of 1534. Element 1535 b may denote the channelused for returning the results of mediated DMA requests of 1534 to theuntrusted fabric 1536.

Elements 1531 a-b denote channels for forwarding DMA requests from theuntrusted fabric 1536 (via the DMA mediation processing of 1534) to thetrusted fabric 1532. In particular, channel 1531 a is a channel forforwarding initial tag read (unvalidated) DMA requests to the trustedfabric 1532 and channel 1531 b is a second channel for forwarding thefinal write of data with tags updated. Use of the two channels maybecome more apparent given further discussion of the DMA or trustbridgepipeline described below in connection with the example 1540. Element1531 c denotes a channel from the trusted fabric 1531 c to the untrustedfabric via the DMA mediation processing 1534.

In one embodiment, element 1534 may represent a DMA processing pipelineas illustrated in the example 1540 of FIG. 77. The example 1540 denotesa 4 stage processing pipeline for servicing a DMA operation as made by aDMA device 1504 a-c from the untrusted or untagged fabric (e.g., 1506 inthe example 1500 and 1536 in the example 1530). Elements 1542, 1544,1546 and 1548 may denote rules triggered as a result of a DMA request.Element 1545 denotes the I/O PUMP such as described in connection withother figures (e.g., 1502 of the example 1500). Element 1543 denotes thestages of the DMW processing pipeline. In a first stage 1541 a, the DMArequest is received from the untrusted fabric and an unvalidated requestis made via rules 1542 in a second memory fetch stage 1541 b to obtainthe requested DMA data and its associated tags from the memory 712 c.The fetched tag information from the memory for the DMA requested datais provided as an input to the third validate stage 1541 c where alookup is performed in the I/O PUMP cache 1545 for a rule correspondingto the current DMA request. If no rule is found in the I/O PUMP, the I/OPUMP processing may be stalled and disabled in stage 1541 c while a rulemiss handler executes in the processor 702 to either calculate theoutputs Rtag and PCnew tag for the DMA request or otherwise determinethat the current DMA request is not allowed (thereby triggering a faultor trap). Assuming that a rule for the current DMA request is located inthe I/O PUMP, it is determined that the DMA request is allowed to beperformed. If the DMA request is a write request, the write data of theDMA request, along with its tag information, is written back to memory712 c in stage 4 1541 d. For DMA write operations, a response 1548 a maybe provided to the untrusted fabric (and then to the DMA device thatinitiated the DMA request) once the write has completed. For DMA readoperations, a response 1546 a may be returned to the untrusted fabric(and then to the DMA device that initiated the DMA request) where theresponse includes the requested data fetched in stage 2 1541 b.

Element 1542 may denote the rules that pass along a request from theuntrusted fabric and pass along information (from stage 1 1541 a)regarding the I/O request for the I/O PUMP 1545 (in stage 3 1541 c)while the memory fetch is performed in stage 2 1541 b. Elements 1544 maydenote the rules that gather up tag responses from the trusted fabric,formulate the actual rule input to the I/O PUMP, and propagateinformation from stage 1541 b to the writeback stage 1541 d to be mergedwith the output of the I/O PUMP.

As a variation to the foregoing embodiment, reference is made back tothe example 1500. In at least one embodiment, rather than have rulesstored in an I/O PUMP cache as described above, the I/O PUMP may beimplemented as a hardwired I/O PUMP where the rules may be implementedusing dedicated hardware such as logic gates wired to embody a fixed setof I/O PUMP load and store DMA rules.

As further variation, the I/O PUMP may alternatively be defined in yetanother embodiment as a cache that is programmable as described inconnection with the example 1500 with the difference that the I/O PUMPas a rule cache has a finite capacity and is filled with a fixed set ofrules that are all stored in the I/O PUMP cache. In this latterembodiment, the I/O PUMP may be populated with the complete set of allDMA rules so that there is never a rule cache miss for the I/O PUMP.Thus, there is never a need to service an I/O PUMP rule cache miss.

What will now be described are techniques that may be used in connectionwith initializing, setting or resetting tags such as may be associatedwith memory locations. Consistent with description elsewhere herein, atag used in connection with such techniques may denote a non-pointer tag(where the non-pointer tag is the actual tag value for the associatedmemory location) or a pointer tag (where the pointer tag is a pointer oraddress of another memory location including the actual tag value orvalues). For example, a pointer tag associated with a memory locationmay be used in connection with composite tags where the pointeridentifies an address in memory including multiple tag values such asfor a plurality of composite policies implemented in parallel. Asdescribed elsewhere herein, example composite policies that may besupported in parallel include a memory safety policy and a control flowintegrity (CFI) policy described elsewhere herein.

Processing performed in connection with memory safety and stackpolicies, for example, may include setting or initializing a largenumber of tags associated with memory locations to a particular value.For example, when allocating a region of memory such as may beassociated with a particular color, each tag associated with a memorylocation in the region needs to be initialized to have the particularcolor value. As another example, when reclaiming a region of memory suchas when freeing the memory region, all memory locations of the freed orunallocated region may be initialized to a particular tag value denotingthe memory locations as free or unallocated.

Processing performed to initialize or reset tags of all memory locationsin a region may consume an unacceptable amount of time and becomesparticularly unacceptable as the size of memory region to be taggedincreases. Thus, described in following paragraphs are techniques thatprovide for efficiently initializing or setting tags (e.g., tagging) ofmemory locations. In at least one embodiment, tag initialization orsetting may be performed, for example, in connection with allocation aregion of memory or freeing a region of memory. Such techniquesdescribed herein are scalable for use with large memory regions.Although such techniques are illustrated below in connection with tagsof memory location, more generally, such techniques may be used inconnection with initializing, setting or resetting values eachassociated with a data item or entity.

In at least one embodiment, the tags and associated memory locations ofa region of memory may be represented in a hierarchical structure orarrangement where the leaves of the hierarchy denote the tags for thememory locations. For purposes of illustration, following discussionmakes reference to a tree as the hierarchical structure. However, moregenerally, any suitable hierarchical structure may be used to representan address space associated with a region of memory locations.

In an extreme case, in one embodiment, the leaves of the tree orhierarchical structure may represent individual words in memory and holdthe tags. However, if an entire subtree is homogeneously tagged with thesame tag value, techniques herein may simply store the tag value at thatparticular node and associated level in the tree without furtherrepresenting any descendant nodes of the subtree. In this case, the tagvalue of the node specifies a tag value for multiple memory locations ofa particular region (e.g., such as a range of consecutive or contiguousmemory addresses). In this manner, storage may be saved in storing tagvalues if there are large homogenously tagged regions. In a worst casescenario where there is no homogeneous tag value (e.g., no two memorylocations having consecutive addresses have the same tag value), theleaves of the tree each represent a tag value for a single memorylocation, such as a single word in the region.

With such a hierarchical structure such as a tree as described infollowing paragraphs, processing may be performed to retag or initializea power-of-two memory region by simply rewriting one node in the tree.For a non-power-of-two region, processing may be performed to partitionthe region into a minimum set of power-of-two regions (e.g., at most2*log₂ (region size) such regions in the minimum set). When a tag of aparticular word or memory location is needed (e.g., read the tag for anassociated memory location), processing may be performed to determinethe tag using the tree. In at least one embodiment described below, ahierarchy of cache memories may be utilized for the different levels ofthe tree. The tag value may be provided by the cache associated with thehighest level in the tree having a cache hit with respect to the desiredmemory location (e.g., perform a cache lookup for the tag value for theaddress of the desired memory location). In connection with processingperformed to write or modify a tag value associated with a memorylocation, processing may include performing a single write to mark asubtree, or multiple writes (e.g., 2*log₂ (region size) log writes).Such multiple writes may be performed, for example, responsive tomodifying or setting the tag of a first memory location included in afirst memory region that is homogenously tagged prior to such modifyingor setting the tag value. In this case, setting or modifying the tagvalue causes the first memory region to no longer be homogenously taggedand the hierarchical structure denoting the tag values for the firstmemory region is accordingly updated to further decompose a subtreedenoting tag values for the first memory region.

Referring to FIG. 78, shown is an example of a hierarchical structurethat may be used to represent tag values for an address spacecorresponding to a region of memory in an embodiment in accordance withtechniques herein. The example 100 illustrates a tree as a hierarchicalstructure used to represent a memory region including 8 memory locationsfor purposes of simplicity of illustration. More generally, techniquesherein may be used to represent tag values for any address space ormemory region using any suitable hierarchy having any number of levels,any suitable number of nodes at each level, any suitable number of childnodes per parent node, and the like.

The example 100 illustrates a binary tree representation of tag valuesfor 8 memory locations having addresses 0 through 7, inclusively. Thetree in this example may include up to 4 levels of nodes, depending onwhich of the 8 memory locations, if any, are homogeneously tagged usinga same subtree of the structure 100. In this example, the entire memoryregion of 8 memory locations may be partitioned repeatedly into power oftwo smaller memory regions where each such partitioning of smallermemory regions corresponds to a different level of nodes in the tree.For example, level 1 104 includes node A1 corresponding to the entireaddress space 0 through 7 which is partitioned into two smaller regionseach represented by a node (nodes B1 and B2) at level 2 106. Level 2 106includes node B1 associated with addresses 0-3 and node B2 associatedwith addresses 4-7.

Each of the nodes B1 and B2 at level 2 106 may be further partitionedinto two smaller regions each represented by a node at level 3 108. NodeB1 and its associated address range 0-3 is partitioned into two regionsrepresented by nodes C1 and C2, where C1 is associated with addressrange 0-1 and C2 is associated with address range 2-3. Similarly, nodeB2 and its associated address range 4-7 is partitioned into two regionsrepresented by nodes C3 and C4, where C3 is associated with addressrange 4-5 and C4 is associated with address range 6-7.

Each of the nodes C1-C4 at level 3 108 may be further partitioned intotwo smaller regions each represented by a node at level 4 110. In thisexample, nodes at level 4 each represent a tag value for a single wordor memory location. Node C1 and its associated address range 0-1 ispartitioned into two regions represented by nodes D1 and D2, where D1 isassociated with address 0 and D2 is associated with address 1. Node C2and its associated address range 2-3 is partitioned into two regionsrepresented by nodes D3 and D4, where D3 is associated with address 2and D4 is associated with address 3. Node C3 and its associated addressrange 4-5 is partitioned into two regions represented by nodes D5 andD6, where D5 is associated with address 4 and D6 is associated withaddress 5. Node C4 and its associated address range 6-7 is partitionedinto two regions represented by nodes D7 and D8, where D7 is associatedwith address 6 and D8 is associated with address 7.

All the nodes A1, B1-B2, C1-C4 and D1-D8 may represent the maximumnumber of possible nodes that may exist in the hierarchicalrepresentation of tag values for the region of 8 memory locations.However, as described in more detail below, the particular nodesincluded in the tree denoting the tag values stored in the memorylocations 0-7 may vary depending on the particular tag values andhomogenous and non-homogeneous tag regions represented at various pointsin time. Levels of the hierarchy may be ranked from a highest levelcorresponding to the root or level 1 104 node to the lowest levelcorresponding to the bottom most level 4 110.

In connection with techniques herein with a first example, reference ismade to 120 of FIG. 79. In this first example, assume all memorylocations associated with a node at a particular level in the hierarchy120 have the same tag value, T1, thereby denoting a subtree ofhomogenously tagged memory locations, the node at the particular levelhas the tag value for all such memory locations and no furtherdescendant nodes in the subtree need to be consulted to determine thetag value for any of the homogenously tagged memory locations. Forexample, if all memory locations 0-7 include the same tag value T1 suchas in connection with initializing the region of memory with the sametag, the tag value T1 for memory locations 0-7 may be stored at node A1(e.g., as denoted by the “tag=T1” indication by node A1). In at leastone embodiment, there is no further need to store additional tag valuesfor other nodes of the tree since the tag values for the entire regionfor addresses 0-7 is represented by the single node A1. In this case,element 122 denotes the single node that may be included in thehierarchical representation of tag values stored at memory locationswith addresses 0-7 and the remaining nodes B1, B2, C1-C4 and D1-D8 maybe omitted from the hierarchical representation.

In a second example, reference is made to 130 of FIG. 80. In this secondexample, assume that memory locations 0-3 have the same first tag valueT1 and memory locations 4-7 have the same second tag value T2 (first andsecond tag values being different). In this case, node A1 may include anindicator (e.g., denoted by the “TAG VALUE=NO TAG VALUE” indication bynode A1) denoting that node A1 does not specify a homogeneous tag formemory locations 0-7 and tag values for memory locations 0-7 arespecified by nodes at one or more lower levels of the hierarchy. Atlevel 2 of the hierarchy, the first tag value, T1, may be stored at nodeB1 (as denoted by the “TAG VALUE=T1 indication by node B1) and thesecond tag value T2 may be stored at node B2 (as denoted by the “TAGVALUE=T2 indication by node B2). The subtree (B1, C1, C2, D1-D4) ofwhich B1 is the root denotes a set of homogenously tagged memorylocations 0-3. The subtree (B2, C3, C4, D5-D8) of which B2 is the rootdenotes another set of homogenously tagged memory locations 4-7. In atleast one embodiment, there is no further need to store additional tagvalues for other nodes of the tree at levels 3 and 4 (e.g., nodes C1-C4for level 3 and nodes D1-D8 for level 4) since the tag values for theentire region for addresses 0-7 are represented by nodes B1 and B2 atlevel 2. In this case, element 132 denotes the nodes that may beincluded in the hierarchical representation of tag values stored atmemory locations with addresses 0-7 and the remaining nodes C1-C4 andD1-D8 may be omitted from the hierarchical representation.

At a first point in time, the tag hierarchy may be as described inconnection with the example 120 with only the single node 122 since alltags have the same tag value. At a subsequent second point in time, tagvalues for addresses 0-3 may be modified to be the same first tag valueT1 and addresses 4-7 may be modified to be the same second tag value T2.As a result of the foregoing tag modifications, two addition nodes B1and B2 as described above in the example 130 may be added to thehierarchy. Assume now at a subsequent third point in time tag values foraddresses 0-3 remain the same as in the example 130. However, tag valuesfor addresses 4-7 may be modified as described below in connection withFIG. 81 whereby additional nodes C3-C4 and D5-D6 are added to the taghierarchy.

In a third example, reference is made to 140 of FIG. 81. In this thirdexample, assume that memory locations 0-3 have the same first tag valueT1 as described above (where the first tag value T1 may be stored atnode B1 and the subtree (B1, C1, C2, D1-D4) of which B1 is the rootdenotes a set of homogenously tagged memory locations 0-3). Further,assume that the memory locations 4-5 each include a different tag valuewhere memory location 4 has tag value T3 and memory location 5 has tagvalue T4, and that memory locations 6-7 are homogeneously tagged andinclude the same tag value T5. In this case, consistent with descriptionabove, nodes A1, B2, and C3 may each include an indicator (e.g., TAG=NOTAG) that the particular node does not specify a tag value whereby nodesat one or more lower levels in the hierarchy specify tag values for theparticular memory locations associated with nodes A1, B2 and C3. Forexample, node C3 corresponding to memory locations 4-5 may include anindicator that the node C3 does not specify a tag value whereby nodes atone or more lower levels in the hierarchy specify tag values for memorylocations 4-5. Node D5 at level 4 may specify the tag value T3 (e.g.,TAG=T3 indicator by node D5) for memory location 4 and node D6 and level4 may specify the tag value T4 (e.g., TAG=T4 indicator by node D6) formemory location 5. Node C4 corresponding to memory locations 6-7 mayspecify the tag value T5 (e.g., TAG=T5 indicator by node C4), thehomogeneous tag value common to memory locations 6-7 and indicate thereis no further need to store additional tag values in nodes D7 and D8(e.g., no need to further examine descendant nodes D7, D8 of C4). Inthis case, element 142 denotes the nodes that may be included in thehierarchical representation of tag values stored at memory locationswith addresses 0-7 and the remaining nodes C1-C2, D1-D4 and D7-D8 may beomitted from the hierarchical representation.

The foregoing illustrations of 120, 130 and 140 may denote thehierarchical representation of tag values for the memory region foraddresses or memory locations 0-7 at different points in time as tagvalues associated with the memory locations may change over time. In amanner similar to adding nodes to the hierarchy as described above,nodes may be removed from the hierarchy as needed as subtrees ofexisting nodes are modified to all have the same tag value (e.g., ifdescendants of a node all have the same tag value then all descendantnodes may be removed from the hierarchy and the node may be used as thesole node of a subtree to denote the single homogeneous tag value of thenode and its descendants).

In at least one embodiment, when a first node at a level in the treespecifies a value for one or more memory locations associated with thefirst node, there is no need to further represent descendant nodes ofthe first node (e.g. no need to further represent nodes of the subtreebeyond the first node). To further illustrate, reference is made to thefirst example noted above in 120 of FIG. 79 where only a tag value ofnode A1 is needed to represent the single homogeneous tag value for thememory region 0-7. To further illustrate, reference is made to the thirdexample 140 of FIG. 81 noted above where an embodiment may not furtherrepresent nodes C1-C2, D1-D4, and D7-D8. In this manner, using such ahierarchical representation of memory locations and associated tagvalues may save storage in connection with tag values for the memorylocations. In other words, in at least one embodiment, rather thanalways allocate and store individual tag values for each of the memorylocations, storage may be allocated where a single tag value in thehierarchy denote a homogeneous tag value for multiple memory locationshaving consecutive or contiguous addresses. With reference to the firstexample noted in 120, rather than allocate storage for 8 tag values formemory locations 0-7 each including the same tag value, memory may beallocated for storing the single tag value of node A1.

In a worst case scenario assuming there are no homogeneously taggedmemory locations in the memory region having addresses 0-7, the entirehierarchical structure of nodes of FIG. 78 is used to represent the tagvalues stored at the addresses 0-7. For example, each of the leaves ofthe hierarchy may represent a different word in memory. Thus, the bottomlevel 4 110 of the hierarchy may denote the tag values for the addressspace 0-7.

Referring back to FIG. 78, assume there is an 8 bit address space usedto represent the addresses of the memory locations 0-7. In at least oneembodiment, the entire 8 bit address space may be partitioned intodifferent memory regions each including 8 memory locations where each ofthe different memory regions may have tag values represented by adifferent instance of a tag value hierarchy. Thus, for the memory regionof addresses 0-7 just described, the highest or top 5 bits are all=0 andthe addresses 0-7 may be represented in the remaining lower 3 bits. Thehighest or top 5 bits=0 may thus be used to indicate the memory regionof addresses 0-7. In such an embodiment, each memory region of 8 memorylocations may have a separate tag value hierarchy such as illustrated inFIG. 78 denoting tag values of the particular memory region. In thisexample, each of the different memory regions denoting a different rangeof 8 addresses or memory locations may be differentiated by examiningthe top 5 bits of the 8 bit address of a memory location.

In at least one embodiment in accordance with techniques herein, aseries of tag cache memories may be used where the number of tag cachesmay correspond to the number of levels in the hierarchy of nodesdenoting tag values. Continuing with the example discussed above andwith reference back to 100 of FIG. 78, each instance of a tag hierarchyfor a memory region of 8 memory locations has 4 levels. In such a case,an embodiment may use 4 tag cache memories 152, 154, 156 and 158 asillustrated in the example 150 of FIG. 82 to store tags for memorylocations. Generally, each of the 4 tag cache memories 152, 154, 156 and158 is associated with a different level in the tag value hierarchy andmay store information about each node in the associated different levelof the tag value hierarchy. For example, tag level cache 152 may includeinformation for level 1 104 nodes or the roots of the tag valuehierarchies for each of the memory regions (which in this particularexample as noted above is a memory region of 8 memory locations). Taglevel cache 154 may include information for level 2 106 nodes of the tagvalue hierarchies for each of the memory regions. Tag level cache 156may include information for level 3 108 nodes of the tag valuehierarchies for each of the memory regions. Tag level cache 158 mayinclude information for level 4 108 nodes of the tag value hierarchiesfor each of the memory regions. The lowest or bottom most level in thehierarchy, which is level 4 110 in this example, may correspond to cachelines for memory locations that may be stored in the data cache (e.g.,denotes as L1-D$ such as denoted by element 20 of FIG. 1). An embodimentmay have a level 4 158 of the tag cache and additionally have metadatatags that may be separately stored in the cache lines of the data cache.Each of the caches 152, 154, 156 and 158 of nodes have an associatedrepresentation in main memory.

In connection with the example embodiment described herein with an 8 bitaddress space, the top or highest 5 bits 152 a of the address of amemory location may be used by level 1 cache 152 to lookup whether thecache 152 includes any level 1 node for the address of the memorylocation. The top or highest 6 bits 154 a of the address of the memorylocation may be used by level 2 cache 154 to lookup whether the cache154 includes any level 2 node for the address of the memory location.The top or highest 7 bits 156 a of the address of a memory location maybe used by level 3 cache 156 to lookup whether the cache 156 includesany level 3 node for the address of the memory location. The 8 bits 154a of the address of the memory location may be used by the level 4 cache158 to lookup whether the cache 158 includes any level 4 node for theaddress of the memory location.

For a particular address, each of the caches associated with a levelother than the bottom most level, may return:

-   -   1). the tag value for the particular address (denoting that this        is a homogeneous tag value at that level for multiple        addresses);    -   2). an indicator that the cache does not specify a tag value for        the particular address and a cache at a lower level in the        hierarchy needs to be consulted to obtain the tag value for the        particular address (this particular level does not specify a        homogeneous tag value for the particular address); or    -   3). null or a second indicator denoting that there is no cache        location in that particular level cache including node or tag        information corresponding to the particular address. The second        indicator also denotes that no cache at a lower non-bottom level        cache includes a node or tag information for the address. This        is discussed below in more detail.

Consistent with discussion above, the indicator returned in item 2)above may be the “NO TAG” indicator associated with a node such asillustrated in examples 120, 130 and 140. For example, with reference tothe illustration 130 of FIG. 80, assume processing is performed todetermine the tag for memory location 5. In this case, level 1 cache 152may return the NO TAG indicator indicating that the tag value for memorylocation 5 is specified by one of the other lower level caches 154, 156,or 158. Level 2 cache 154 may return the tag value T2 for memorylocation 5 illustrating returned cache item 1) above. To illustratereturned item 3) above where the second indicator is returned, considerthe level 3 cache 156. The level 3 cache 156 may not include any nodeinformation corresponding to the memory location 5 (e.g., no cachelocation includes node or tag information associated with memorylocation 5 lookup) and so the second indicator described above in 3) maybe returned indicating that there is no node information for the memorylocation 5 in this level 3 cache. In such an embodiment, processing maygenerally utilize the tag value returned from the highest tag levelcache. For example, in connection with memory location 5 with referenceto the example 130, level 2 cache 154 is the highest level of the tagcaches returning a tag value for memory location 5.

For contents of a memory location stored in an L1 (level 1) data cache,cached information may include the current tag value and also the levelin the tag cache hierarchy where the tag value is defined. Referringagain to the example above for the memory location 5 using the hierarchy130 of FIG. 80, if the contents of memory location 5 is also stored inthe data cache, the data cache may include the tag value T2 and alsoinformation that the current tag value T2 is defined by a level 2 node(e.g. B2) having its node information stored in the level 2 cache 154.Thus, the example 150 illustrates 4 tag caches in the tag cachehierarchy where tag values may be stored and the embodiment mayadditionally include a tagged data cache (e.g., L1 data cache) separatefrom any tag information stored in the tag cache hierarchy.

In an embodiment in accordance with techniques herein, processing may beperformed by the PUMP to resolve or determine a tag value for aparticular memory location having a particular address. When performingprocessing to obtain a tag value and contents for a particular memorylocation, there may be a data cache hit whereby the memory locationcontents and its tag are stored in the data cache. Upon the occurrenceof a data cache hit for a memory location, processing may be performedto consult the stored level of the tag cache hierarchy which defines thetag value for this memory location to make sure the first cached tagvalue obtained from the tag caching level matches the second tag valueof the memory location as stored in the data cache. If the two do notmatch, this indicates that the second cached tag value as stored in thedata cache is stale, out of date and has been modified. In this case ifthe second cached tag value stored in the data cache for the memorylocation and the first tag value as obtained from the tag cache for thememory location do not match, processing may be performed includingupdating the second cached tag value as stored in the data cache for thememory location (e.g., to match that as stored in the tag cachehierarchy). In at least one embodiment, for a memory location having itsdata and thus its tag cached in the data cache, information may betracked in the data cache for the memory location's tag including thelevel of the tag cache hierarchy where the tag is defined. The foregoingstoring of the level in the cache tag hierarchy may be an optimizationwhereby the stored level may be used to readily access the tag valuefrom the tag hierarchy (e.g., rather than having to consult all taglevel caches or otherwise search the hierarchy of existing nodes such asin a search from the root or top of hierarchy downward toward the leafnodes). Thus, upon the occurrence of a data cache hit and where the tagvalue stored in the data cache for a memory location does not match thetag value stored in the tag hierarchy for the memory location,processing may include updating the tag value as stored in the datacache and additionally updating the hierarchy level information storedin the data cache as to where the memory location's tag is defined inthe tag hierarchy. Subsequently, processing performed by the PUMP forresolving or determining the tag value for the memory location may berestarted.

Upon the occurrence of a data cache miss for a memory location (e.g.,where the memory location contents and tag are not found in the datacache), processing may be performed to perform a tag cache lookup forthe tag value in levels of the tag cache (e.g., other than the bottommost tag cache level) in parallel. For example, a lookup for the tagvalue for the memory location may be performed by consulting the 4caches 152, 154, 156 and 158, respectively, for levels 1, 2, 3 and 4 ofthe tag caches in parallel. As discussed above, the tag value returnedby the highest level of the tag caches 152, 154, 156 and 158 is used asthe tag value for the memory location. Additionally, it should be notedthat in a properly represented tag value hierarchy, only a single one of152, 154,156 and 158 may return a tag value for the memory location. Inat least one embodiment, the caches 152, 154, 156 and 158 may be indexedto allow for parallel access.

An embodiment may also not perform a parallel lookup or search for aparticular memory location's tag with respect to all 4 tag caches 152,154, 156 and 158. As a variation an embodiment may traverse the tagcaches of the hierarchy from the root node level (level 1) downwardtoward the leaf nodes (e.g., level 4). For a tag cache miss at a levelN, the tree or hierarchy may be traversed inserting nodes into thedifferent levels of tag caches for the particular memory access. Inconnection with a level cache miss for a parallel search of the tagcaches, an embodiment may choose to only insert nodes into the levelcaches when there is a tag. So as some level cache provides a tag, it isnot required that all other level caches have a NO TAG entry.

As discussed elsewhere herein, a tag of a memory location may bemodified. In response to modifying a tag of a memory location,processing may be performed to accordingly update the hierarchyspecifying a tag value for the memory location. Such updating mayinclude invalidating any one or more levels of the hierarchy which areno longer homogeneous. Additionally, processing may be performed toaccordingly update the levels of caches such as illustrated in theexample 150 of FIG. 82.

When performing an operation to set or initialize a tag of a memorylocation, such processing may include the PUMP checking for the validityof performing the desired operation. For example, consider the case withretagging all memory locations in the memory region with a new tag.Processing may include obtaining the current tag values of all memorylocations in the region and checking via PUMP processing for validity ofthe retag. The processing may include clearing the tags of the memorylocations in the region, if allowed, and then updating the tag valuesfor the memory locations in the region, if allowed. Consistent withdiscussion above, updating, modifying, or setting tags of a memoryregion may include accordingly modifying the hierarchy and associatednodes to reflect the tag values for different memory locations in thememory region (e.g., decomposing portions of the region which arehomogeneous prior to the modification and non-homogeneous after themodification whereby there may be additional children or descendantnodes added to reflect the tag value modification(s)).

In at least one embodiment, the hierarchical representation of tagvalues for a memory region may be a tree. For example, the tree may be abinary tree where each node has either 0, 1 or 2 children. As avariation, the hierarchical representation may be a tree but not abinary tree. For example, each node in the tree may be allowed to haveany suitable number of child nodes up to a specified maximum. Thehierarchical representation may include any suitable number of levels,nodes at each level, children per parent node, and the like. As known inthe art, there is a tradeoff between various parameters of thehierarchical representation such as depth or number of levels and nodesat each level/number of children per parent node). For example, thelarger the number of nodes at each level, the fewer the number of levelsand thus the shorter amount of time/levels to be consulted whendetermining a tag value for a memory location. However, in such a case,more writes need be performed to clear a region.

What will now be described are techniques that may be performed, such asby rules triggered as a result of loader code, in connection with a CFIpolicy in an embodiment in accordance with techniques herein. To enforcethe CFI policy using metadata processing rules accessing taginformation, information regarding allowable control flow needs to becommunicated to the metadata processing domain. To this end, anembodiment in accordance with techniques herein may use an approachdescribed in following paragraphs. Generally, a transfer of control ismade from a branch source to a target or destination. In connection withallowable control flow, for a particular control flow target ordestination, a set of sources that are allowed to transfer control tothe particular control flow target or destination may be identified. Theset of sources for each possible control flow target may be communicatedto the metadata processing domain, such as stored metadata taginformation, which may then be used by rules of the CFI policy inconnection with CFI policy enforcement during runtime execution of usercode (e.g., code executing in the code execution domain or non-metadataprocessing domain).

The processing performed may include uniquely tagging each source andthen tagging each target with the set of allowable sources (e.g.,address of the sources) permitted to transfer control to that particulartarget. For example, reference is made to FIG. 83. In the example 1700,element 1701 may denote a code portion of instructions of an applicationexecuted in the code execution or non-metadata processing domain.Element 1702 a and 1704 a-c denote locations of instructions in the codeportion. Element 1702 a denotes a control flow target A. Elements 1704a-c denote control flow sources that are allowed to transfer control totarget A 1702 a. Such transfer of control from each of 1704 a-c isdenoted by the JMP (jump) A instruction. Element 1706 denotes the set ofallowable sources that are permitted to transfer control to the targetA. D7 denotes the unique source tag of instruction 1704 a. C3 denotesthe unique source tag of instruction 1704 b. E8 denotes the uniquesource tag of instruction 1704 c. As illustrated by 1710, the JMP (jump)instructions 1704 a-c are tagged, respectively, as D7, C3 and E8. Asalso illustrated by 1710, the instructions 1704 a-c are also stored,respectively, at addresses 1020, 1028 and 1034. The target location Ahas address 800. In this case, the set of allowable sources, oraddresses of the source instructions allowed to transfer control to thetarget A, may be the set {1020, 1028,1034} denoted by 1706. Thus, theset 1706 is an example of the allowable control flow information thatneeds to be communicated to the metadata processing domain where suchallowable control flow information is stored as tag metadata for use byrules of the CFI policy. In at least one embodiment in accordance withtechniques herein, code of the loader may fire rules that performprocessing to collect the control flow information needed by themetadata processing domain to enforce the CFI policy for the applicationinclude code portion 1701. The loader code may be executed in connectionwith loading the application (e.g., loading executable code for theapplication) whereby the loader code, while executing to load theapplication, triggers rules that perform the necessary processing tocollect the control flow information (as subsequently used by themetadata processing to enforce the CFI policy during execution of theapplication).

In at least one embodiment consistent with description elsewhere herein,execution of kernel code may trigger rules that tag code of the loaderwith special instruction tags enabling the tagged instructions of theloader, when executed, to trigger rules that generate a sequence ofsource tags (each tag of the sequence being unique) used to tag thesources (e.g., generate source tags D7, C3 and E8). For example,reference is made to FIG. 84 including logical processing performed byrules fired as a result of executing code of the loader. The logicalprocessing is described in 1720 using a C-like pseudo code descriptionwhere such processing may be performed for each control flow target suchas A 1702 a. At step 1721, the source set is initialized to the emptyset. At step 1722, for each source that is allowed to transfer controlto the target, steps 1723, 1724 and 1725 may be performed. At step 1723,t is assigned a newly allocated CFI source tag. At step 1724, a sourcelocation (of an instruction that transfers control to the target) istagged with the newly allocated tag t generated in step 1723. In step1725, the source set is updated to also include tag t. In one aspect,the operation of step 1725 may be characterized as forming a set unionof allowable sources for the target where the union operation isperformed in 1725 for each iteration of the loop processing, that beginsat 1722, as performed for each source. Step 1726 tags or marks thetarget with the source set.

Element 1723 may be the following instruction included in loader codethat triggers a rule that generates or allocated the new CFI source tag:

-   -   ADD R1←R1+R1        where the ADD instruction (e.g., such as ADDI in the RISC-V        instruction set) has been tagged by the kernel code with the        special CI tag of CFI-alloc-tag marking this instruction as an        allowable tag generator instruction. In at least one embodiment        a different sequence of source tags may be generated by the        loader for each application in connection with a CFI policy        (e.g., in the example 1620, the loader may use a different        sequence of CFI tags 1630 as the unique sequence of CFI source        tags for an application where the sequence of CFI tags may be        generated from a particular one of the CFI tag generator App-n        tags of 1627). The CFI-alloc-tag is the CI tag placed on the        loader ADD instruction above denoting that the ADD instruction        is allowed to allocate or generate a next tag in the application        specific CFI sequence. CFI-alloc-tag may be one of the special        instruction types of 1624 as described in connection with the        example 1620. The ADD instruction above indicates that the tag        on R1 holds the state of the CFI sequence where the state may be        the last tag of the sequence previously generated. Execution of        the above ADD instruction triggers a rule that generates the        next new tag in the CFI sequence and updates the tag on R1 to        now be the newly generated tag. Using rule conventions as        described elsewhere herein, the following ADD rule may denote        the rule triggered by the above ADD instruction:    -   ADD: (-, CFI-alloc-tag, t1, t1, --)→(-, tlnext)        which ensures that the CI tag for the ADDI instruction is        CFI-alloc-tag. In this ADD rule, t1 denotes the previous tag in        the sequence (saved as the current state of the CFI tag sequence        for the application) which is used to generate the next tag,        t1next, in the sequence, where t1next is then stored as the tag        for RD (the destination or result register). The foregoing tag,        t1next, in the CFI sequence may be used as the unique CFI source        tag placed on a source point.

Element 1724 may be an instruction of the loader code, such as a ST(store) instruction below, used to trigger a rule that tags a sourcelocation with the unique CFI source tag:

-   -   ST R1→R3        where R3 is a pointer to the control flow source location in the        user program code (e.g., 1704 a in the example 1700) being        tagged, and the tag on R1 is the unique CFI source tag to be        placed on the source location. The above ST instruction may also        be tagged with a special CI tag, such as CI-LDR, denoting the ST        instruction is included in loader code triggering rule ST below:    -   ST: (-, CI-LDR, t1, -, codetag)→(-, t1)        where CI tag=CI-LDR, t1 is the CFI source tag currently stored        as the tag on R1, and codetag is an instruction tag on the        source location at address R3 (e.g., ensuring that the source        location is currently tagged as code). As a result, the        destination (R3) is tagged with t1, the unique CFI source tag.

Element 1725 may be an instruction of the loader code, such as an ADDinstruction below, used to trigger a rule that adds the address of thesource (e.g., currently pointed to by R3, where R3 contains the addressof the source) to an accumulated set of CFI source tags denotingallowable source locations that can transfer control to the target:

-   -   ADD R2←R2+R3        where the tag on R2 points to a memory location denoting the        accumulated set of allowable source locations. The above ADD        instruction may be tagged with a special CFI UNION instruction        tag denoting that this ADD instruction is performing a union        operation of CFI sources and the union is stored as a tag on R2.        The following rule for ADD may be fired as a result of the above        ADD instruction:    -   ADD: (-, CFI UNION, tset, tsrc, -)→(tunion)        which checks to ensure that the CI tag is CFI UNION, tset is a        target set, and tsrc is a source tag. It produces a new CFI set,        tunion, that represents the addition of tsrc to tset.

Element 1726 may be an instruction of the loader code, such as a STinstruction below, used to trigger a rule that tags the target with theunion or accumulated list of allowable source locations that cantransfer control to the target:

-   -   ST R2→R17        R17 may be a register containing the address of the target        location, and R2 may be, as noted above, a register tagged with        the current accumulated set union of allowable source locations        (e.g., tag on R2 denotes the set of allowable source locations        for the target location whose address is contained in R17). The        above ST instruction may be tagged with a special instruction        tag CFI MARK TARGET denoting the instruction as a special one        that is allowed to tag a control transfer target location (e.g.,        this STORE instruction 1726 of the loader code may have been        tagged by kernel code in a manner similar to other code tags on        load code instructions that trigger rules to perform processing        of 1720). The following ST rule may be triggered as a result of        the above STORE instruction for 1726:    -   ST: (-, CFI MARK TARGET, tset,-, codetag)→(-, tset)        which triggers when CI tag is CFI MARK TARGET and the target        (pointed to by R17, wherein R17 includes the target address) is        tagged with codetag indicating an instruction, and places the        tset annotation onto the target.

Different tag structures or layouts that may be defined for use withsources, targets and the set of allowable source locations are describedelsewhere herein as well as any other suitable structure definition(e.g., see examples 240, 250, 260, 267, 270 and 280 describing taglayouts for use with tagged source and target locations that may be usedmore generally with any instruction as well as in connection withmultiple instructions per tagged word).

Thus, the processing steps described above as in the example 1720 may beperformed by having code of the loader properly tagged so that when suchloader code is executed, rules are fired which cause the steps of theexample 1720 to be performed by the metadata processing domain in anembodiment in accordance with techniques herein. It should be noted thatthe foregoing sequence of instructions and fired rules as a result ofthe instructions are merely one examples of instructions and rules thatmay be used in an embodiment using techniques herein. For example, anembodiment may include a different instruction other than an ADD inloader code that triggers a rule performing processing as describedabove (e.g., element 1725).

In the foregoing description, certain terms have been used for brevity,clearness, and understanding. No unnecessary limitations are to beimplied therefrom beyond the requirement of the prior art because suchterms are used for descriptive purposes and are intended to be broadlyconstrued. Moreover, the description and illustration of the preferredembodiment of the present disclosure are an example and the presentdisclosure is not limited to the exact details shown or described.

Various aspects of techniques described herein may be performed byexecuting code which is stored on any one or more different forms ofcomputer-readable media. Computer-readable media may include differentforms of volatile (e.g., RAM) and non-volatile (e.g., ROM, flash memory,magnetic or optical disks, or tape) storage which may be removable ornon-removable.

While the invention has been disclosed in connection with variousembodiments shown and described in detail, their modifications andimprovements thereon will become readily apparent to those skilled inthe art. Accordingly, the spirit and scope of the present inventionshould be limited only by the following claims.

What is claimed is:
 1. A method comprising: at an input/output (IO)metadata processor having a cache configured to store one or more rules,receiving, from a first fabric, a direct memory access (DMA) requestdirected to an address of a second fabric; generating an unvalidatedrequest based on the DMA request and at least one of the one or morerules; obtaining DMA data requested by the unvalidated request at theaddress of the second fabric and obtaining tags associated with the DMAdata; attempting to load at least one of the one or more rules from thecache of the IO metadata processor based on the tags obtained; if the atleast one rule based on the tags obtained is not found in the cache,executing rule miss handling; and if the at least one rule based on thetags obtained is found in the cache, allowing the DMA request.
 2. Themethod of claim 1, wherein the first fabric is an untrusted fabric, andthe second fabric is a trusted fabric.
 3. The method of claim 2, furthercomprising: if the DMA request is allowed, allowing reading of theaddress of the trusted fabric; and if the DMA request is allowed and theDMA request is a write request, allowing write back to the address ofthe trusted fabric.
 4. The method of claim 3, wherein allowing readingof the address and allowing write back is performed over an outputchannel between the IO metadata processor and the untrusted fabric. 5.The method of claim 1, wherein the at least one rule based on the tagsindicates a set of addresses.
 6. The method of claim 2, furthercomprising storing device state information in an untrusted fabricdevice register file.
 7. The method of claim 2 wherein obtaining the DMAdata is over an initial channel from the trusted fabric to the IOmetadata processor, the channel being private with respect to theuntrusted fabric.
 8. The method of claim 2, wherein the untrusted fabricis at least one of a network device, ethernet DMA device, universalasynchronous receiver/transmitter (UART), and serial communicationdevice.
 9. The method of claim 2, wherein executing rule cache misshandling includes at least one of (i) rejecting the DMA request and (ii)redirecting the DMA request to an address separate from the trustedfabric.
 10. The method of claim 1, wherein executing rule cache misshandling includes sending, from the IO metadata processor, an interruptto a processor connected with the IO metadata processor.
 11. The methodof claim 2, wherein the DMA data includes at least one of an operationgroup, a tag identifying a state of the untrusted fabric, a tagidentifying the untrusted fabric, a tag indicating a trust level of theuntrusted fabric, a memory input tag, an enable byte tag, a memoryresult tac, and tag of a state of the untrusted fabric state after theDMA request is processed.
 12. A system comprising: an input/output (TO)cache; and an input/output (TO) metadata processor having a cacheconfigured to store one or more rules, the IO metadata processorconfigured to: receive, from a first fabric, a direct memory access(DMA) request directed to an address of a second fabric; generate anunvalidated request based on the DMA request and at least one of the oneor more rules; obtain DMA data requested by the unvalidated request atthe address of the second fabric and obtain tags associated with the DMAdata; and attempt to load at least one of the one or more rules from theTO cache of the TO metadata processor based on the tags obtained; if theat least one rule based on the tags obtained is not found in the TOcache, execute rule miss handling; if the at least one rule based on thetags obtained is found in the TO cache, allowing the DMA request. 13.The system of claim 12, wherein the first fabric is an untrusted fabric,and the second fabric is a trusted fabric.
 14. The system of claim 13,wherein the TO metadata processor is further configured to: if the DMArequest is allowed, allow reading of the address of the trusted fabric;and if the DMA request is allowed and the DMA request is a writerequest, allow write back to the address of the trusted fabric.
 15. Thesystem of claim 14, wherein allowing reading of the address and allowingwrite back is performed over an output channel between the TO metadataprocessor and the untrusted fabric.
 16. The system of claim 12, whereinthe rule based on the tags indicates a set of addresses.
 17. The systemof claim 13, wherein the TO metadata processor is further configured tostore device state information in an untrusted fabric device registerfile.
 18. The system of claim 13 wherein the TO metadata processor isconfigured to obtain the DMA data over an initial channel from thetrusted fabric to the TO metadata processor, the channel being privatewith respect to the untrusted fabric.
 19. The system of claim 13,wherein the untrusted fabric is at least one of a network device,ethernet DMA device, universal asynchronous receiver/transmitter (UART),and serial communication device.
 20. The system of claim 13, whereinexecution of rule cache miss handling includes at least one of (i)rejection of the DMA request and (ii) redirection of the DMA request toan address separate from the trusted fabric.
 21. The system of claim 13,wherein execution of rule cache miss handling includes conveyance, fromthe IO metadata processor, of an interrupt to a processor connected withthe IO metadata processor.
 22. The system of claim 13, wherein the DMAdata includes at least one of an operation group, a tag identifying astate of the untrusted fabric, a tag identifying the untrusted fabric, atag indicating a trust level of the untrusted fabric, a memory inputtag, an enable byte tag, a memory result tac, and tag of a state of theuntrusted fabric state after the DMA request is processed.
 23. A methodcomprising acts of: receiving a direct memory access (DMA) request toaccess data at a memory address, wherein the memory address hasassociated therewith a metadata tag; obtaining one or more metadata tagsassociated with the DMA request, the one or more metadata tagscomprising the metadata tag associated with the memory address; usingthe one or more metadata tags associated with the DMA request to query arule cache storing one or more rules of a DMA policy; and determiningwhether to allow the DMA request, based at least in part on whether theone or more metadata tags associated with the DMA request match any rulein the rule cache.
 24. The method of claim 23, wherein the act ofdetermining whether to allow the DMA request comprises an act of: inresponse to determining that the one or more metadata tags associatedwith the DMA request does not match any rule in the rule cache,triggering an interrupt to be serviced by a processor.
 25. The method ofclaim 23, further comprising an act of: in response to the processorreturning an indication that the DMA request is allowed, inserting arule into the rule cache, based at least in part on the one or moremetadata tags associated with the DMA request.
 26. The method of claim25, wherein: the one or more metadata tags associated with the DMArequest comprise one or more input metadata tags; the processor returnsone or more output metadata tags; and the rule inserted into the rulecache is further based on the one or more output metadata tags.
 27. Themethod of claim 23, wherein the one or more metadata tags associatedwith the DMA request further comprise at least one metadata tag selectedfrom a group consisting of: a metadata tag indicating a type of the DMArequest; a metadata tag identifying a device from which the DMA requestis received; a metadata tag indicating a state of the device from whichthe DMA request is received; and a metadata tag associated with aregister for storing data loaded from, or to be stored to, the memoryaddress pursuant to the DMA request.
 28. The method of claim 23,wherein: the one or more metadata tags associated with the DMA requestcomprise one or more input metadata tags; the one or more metadata tagsassociated with the DMA request match a rule in the rule cache; and themethod further comprises an act of: determining, based at least in parton the matched rule, one or more output metadata tags.
 29. The method ofclaim 26, wherein: the DMA request is received from a device; themetadata tag associated with the memory address comprises a metadata tagassociated with the memory address before the DMA request is serviced;and the one or more output metadata tags comprise at least one metadatatag selected from a group consisting of: a metadata tag indicating astate of the device after the DMA request is serviced; and a metadatatag to be associated with the memory address after the DMA request isserviced.
 30. The method of claim 23, wherein the rule cache comprises arule indicating that a selected device is allowed to access one or moreselected memory ranges.
 31. The method of claim 30, wherein: the DMArequest is received from the selected device; the memory address is inthe one or more selected memory ranges; the one or more metadata tagsassociated with the DMA request further comprise a metadata tagassociated with the selected device; and the metadata tag associatedwith the selected device and the metadata tag associated with the memoryaddress match the rule indicating that the selected device is allowed toaccess the one or more selected memory ranges.
 32. The method of claim23, wherein the DMA request comprises a request to transfer data betweena trusted domain and an untrusted domain.
 33. The method of claim 31,wherein: the trusted domain comprises a trusted fabric; and theuntrusted domain comprises an untrusted fabric.
 34. The method of claim23, wherein the act of obtaining one or more metadata tags associatedwith the DMA request comprises acts of: identifying, from a deviceregister file, an entry corresponding to a device from which the DMArequest is received; and obtaining, based at least in part on theidentified entry, at least one metadata tag selected from a groupconsisting of: a metadata tag identifying the device from which the DMArequest is received; and a metadata tag indicating a state of the devicefrom which the DMA request is received.
 35. The method of claim 23,wherein the DMA request comprises at least one request selected from agroup consisting of: a request to load data from the memory address; anda request to store data to the memory address.
 36. A method comprisingacts of: receiving a direct memory access (DMA) request to access dataat a memory address, wherein the memory address has associated therewitha metadata tag; obtaining one or more metadata tags associated with theDMA request, the one or more metadata tags comprising the metadata tagassociated with the memory address; and determining whether to allow theDMA request, comprising: using hardware logic to process the one or moremetadata tags associated with the DMA request, wherein the hardwarelogic implements one or more rules of a DMA policy.